Amos 1 Large
Explore Amos 1 Large: exact decision configuration, published specifications, source-linked benchmarks and API tariffs when available.
← All decision models · Compare this decision system →
439 recorded observations. Published results retain their source and setup; no Examenos evaluation was run. Missing results are not zero. How to read these comparisons →
Evaluator-published · etsabary System 1 Decision Benchmark
| Evaluation | Result | Evidence |
|---|---|---|
| System 1 Decision Benchmark v4 · All-question accuracy | 73.667% | Setup & sourceCorrect answers over all 2,157 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 2157 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · Macro-category accuracy | 76.005% | Setup & sourceEqual-weight mean over the 43 categories. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 2157 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · Added 140 AQuA questions | 52.143% | Setup & sourceAccuracy on the 140 questions new in v4. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 140 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · Accuracy among usable responses | 73.667% | Setup & sourceCorrect answers over usable responses only; shown beside coverage. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 2157 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · Usable-response coverage | 100.000% | Setup & sourceShare of the 2,157 inputs with a usable response. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 2157 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · Median client time | 327.4 ms | Setup & sourcePooled primary client time, network and failed requests included. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 2157 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · P95 client time | 912.2 ms | Setup & sourcePooled primary client time, network and failed requests included. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 2157 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F01 · Lookup and filtering | 93.023% | Setup & sourceLookup and filtering: 43 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 43 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F02 · Forward conditionals | 95.652% | Setup & sourceForward conditionals: 46 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 46 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F03 · Necessary conditions | 94.231% | Setup & sourceNecessary conditions: 52 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 52 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F04 · Boolean scope | 94.444% | Setup & sourceBoolean scope: 54 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 54 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F05 · Quantifiers and sets | 91.837% | Setup & sourceQuantifiers and sets: 49 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 49 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F06 · Comparisons | 95.000% | Setup & sourceComparisons: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F07 · Scheduling | 71.053% | Setup & sourceScheduling: 76 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 76 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F08 · One-to-one assignments | 92.537% | Setup & sourceOne-to-one assignments: 67 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 67 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F09 · Exact truth counting | 52.113% | Setup & sourceExact truth counting: 71 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 71 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F10 · Truth-teller consistency | 68.919% | Setup & sourceTruth-teller consistency: 74 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 74 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F11 · Sufficiency and inconsistency | 85.366% | Setup & sourceSufficiency and inconsistency: 41 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 41 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F12 · Object swaps | 50.000% | Setup & sourceObject swaps: 52 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 52 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F13 · Sequential procedures | 33.333% | Setup & sourceSequential procedures: 48 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 48 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F14 · Temporal intervals | 97.619% | Setup & sourceTemporal intervals: 42 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 42 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F15 · Spatial tracking | 73.469% | Setup & sourceSpatial tracking: 49 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 49 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F16 · Reference binding | 60.000% | Setup & sourceReference binding: 50 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 50 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F17 · Beliefs and perspectives | 92.000% | Setup & sourceBeliefs and perspectives: 50 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 50 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F18 · Causal intervention | 72.000% | Setup & sourceCausal intervention: 50 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 50 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F19 · Exceptions and priorities | 94.828% | Setup & sourceExceptions and priorities: 58 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 58 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F20 · Planning | 88.372% | Setup & sourcePlanning: 43 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 43 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F21 · Directed reachability | 95.349% | Setup & sourceDirected reachability: 43 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 43 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F22 · Preference ordering | 92.157% | Setup & sourcePreference ordering: 51 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 51 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F23 · Rule counterexamples | 96.154% | Setup & sourceRule counterexamples: 52 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 52 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F24 · Authority and stale records | 95.349% | Setup & sourceAuthority and stale records: 43 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 43 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F25 · Fault diagnosis | 88.000% | Setup & sourceFault diagnosis: 50 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 50 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F26 · Probabilistic risk | 65.000% | Setup & sourceProbabilistic risk: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F27 · Value of information | 62.500% | Setup & sourceValue of information: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F28 · Robust decisions without a prior | 35.000% | Setup & sourceRobust decisions without a prior: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F29 · Strategic decisions | 57.500% | Setup & sourceStrategic decisions: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F30 · Concurrent-history consistency | 57.500% | Setup & sourceConcurrent-history consistency: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F31 · Information-flow security | 70.000% | Setup & sourceInformation-flow security: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · P01 · Understanding requests and routing | 87.500% | Setup & sourceUnderstanding requests and routing: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · P02 · Recognizing multiple requested actions | 95.000% | Setup & sourceRecognizing multiple requested actions: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · P03 · Tracking current conversational intention | 97.500% | Setup & sourceTracking current conversational intention: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · P04 · Selecting a tool or requesting clarification | 50.000% | Setup & sourceSelecting a tool or requesting clarification: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · P05 · Checking authorization and scope | 57.500% | Setup & sourceChecking authorization and scope: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · P06 · Making decisions with untrusted content | 92.500% | Setup & sourceMaking decisions with untrusted content: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · P07 · Applying a contextual content policy | 75.000% | Setup & sourceApplying a contextual content policy: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · P08 · Extracting and normalizing facts | 47.500% | Setup & sourceExtracting and normalizing facts: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · P09 · Checking claims against supplied evidence | 65.000% | Setup & sourceChecking claims against supplied evidence: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · P10 · Assigning severity or priority from a rubric | 75.000% | Setup & sourceAssigning severity or priority from a rubric: 40 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 40 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F32 · Numerical pattern recognition | 73.913% | Setup & sourceNumerical pattern recognition: 23 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 23 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
| System 1 Decision Benchmark v4 · F33 · Quantitative reasoning | 41.500% | Setup & sourceQuantitative reasoning: 200 questions. Amos 1 Large; Hosted native; collected 2026-10-20 to 2026-11-02 America/Toronto (4 days); one primary response per input, no scored retries. Identity: Captured serving declaration and returned aliases; immutable hidden weights not independently attested. Hosted free access; its responses are identified by serving declaration only. Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here. Checked / reported: 2026-11-08 · Samples: 200 Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established) Result source · Benchmark source |
Official / vendor · Scira (vendor run / card)
| Evaluation | Result | Evidence |
|---|---|---|
| Scira · Amos 1 cards · Typed decisions · accuracy | 76.1% | Setup & sourceTyped decisions test split, 2,000 decisions from four workflows, zero-shot. Amos 1 large, card d67e051. Typed decisions test split, zero-shot. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Typed decisions · KL | 0.126 KL | Setup & sourceTyped decisions test split, 2,000 decisions from four workflows, zero-shot. Amos 1 large, card d67e051. Typed decisions test split, zero-shot. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Typed decisions · Brier | 0.069 Brier | Setup & sourceTyped decisions test split, 2,000 decisions from four workflows, zero-shot. Amos 1 large, card d67e051. Typed decisions test split, zero-shot. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Typed decisions · accuracy with priors fitted on train | 80.25% | Setup & sourcePer-question priors fitted on the benchmark’s train split and applied at answer time; the weights do not change. Not zero-shot. Amos 1 large, card d67e051. Priors fitted on the train split; not zero-shot. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Decision Bench · accuracy | 94.9% | Setup & sourceDecision Bench, all 1,071 real records, images included, with the benchmark’s own harness. Amos 1 large, card d67e051. All 1,071 records with the benchmark’s harness. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: 1071 Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Decision Index 0.3 · public part | 65.11 | Setup & sourceDecision Index 0.3 public part, scored with the board’s own kit; the board’s ranking adds private tests. Amos 1 large, card d67e051. Public part with the board’s kit. The model page printed 64.94; the pinned card value is stored. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · JevBench public · easy | 100.0% | Setup & sourceJevBench public easy items, as served. Amos 1 large, card d67e051. As served. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · JevBench public · standard | 100.0% | Setup & sourceJevBench public standard items, as served. Amos 1 large, card d67e051. As served. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · JevBench public · hard | 84.7% | Setup & sourceJevBench public hard items, as served. Amos 1 large, card d67e051. As served. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Ten public benchmarks | 89.8% | Setup & sourceMean over RAGTruth, JudgeBench, Belebele, WinoGrande, ContractNLI, TruthfulQA, Circa, Financial PhraseBank, TabFact and BBH, as served. Amos 1 large, card d67e051. As served. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · AG News | 89.1% | Setup & sourceAG News, as served. Amos 1 large, card d67e051. As served. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · DAIR Emotion | 58.5% | Setup & sourceDAIR Emotion, as served. Amos 1 large, card d67e051. As served. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Banking77 | 90.5% | Setup & sourceBanking77, as served. Amos 1 large, card d67e051. As served. 77 labels. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Invoice totals quoted exactly | 10 of 10 | Setup & sourceInvoice totals quoted exactly, out of 10, as served. Amos 1 large, card d67e051. As served. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: 10 Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · RefCOCO · box centred on the named object | 91.9% | Setup & sourceShare of RefCOCO boxes centred on the named object, as served. Amos 1 large, card d67e051. As served. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · RefCOCO · box covering at least half the object | 71.3% | Setup & sourceShare of RefCOCO boxes covering at least half the named object, as served. Amos 1 large, card d67e051. As served. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · COCO · box centred on the named object | 98.4% | Setup & sourceShare of COCO boxes centred on the named object, as served. Amos 1 large, card d67e051. As served. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Copy of the Decision Index vision board’s public tasks | 76.69 | Setup & sourceScira’s copy of the Decision Index vision board’s public tasks (8 tasks, board weights); not the board’s own run. Amos 1 large, card d67e051. Scira’s copy, 8 tasks, board weights. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · BLINK · chance-corrected | 65.9 | Setup & sourceBLINK, chance-corrected, as served. Amos 1 large, card d67e051. As served. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Reranking the BM25 top 30 · nDCG@10 | 0.517 nDCG@10 | Setup & sourceReranking the BM25 top 30, nDCG@10, mean of SciFact, NFCorpus and FiQA. Amos 1 large, card d67e051. Mean of SciFact, NFCorpus and FiQA. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Server time, median, three-question request | 122 ms | Setup & sourceMedian server time for a three-question request; hardware not stated. Amos 1 large, card d67e051. Median, three-question request; hardware not stated. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Decision Index 0.3 · public part · knowledge | 52.6 | Setup & sourceDecision Index 0.3 public part, knowledge area, with the board’s own kit. Amos 1 large, card d67e051. Public part with the board’s kit; area score. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Decision Index 0.3 · public part · language | 72.3 | Setup & sourceDecision Index 0.3 public part, language area, with the board’s own kit. Amos 1 large, card d67e051. Public part with the board’s kit; area score. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Decision Index 0.3 · public part · retrieval | 66.2 | Setup & sourceDecision Index 0.3 public part, retrieval area, with the board’s own kit. Amos 1 large, card d67e051. Public part with the board’s kit; area score. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Decision Index 0.3 · public part · tools | 79.7 | Setup & sourceDecision Index 0.3 public part, tools area, with the board’s own kit. Amos 1 large, card d67e051. Public part with the board’s kit; area score. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
| Scira · Amos 1 cards · Decision Index 0.3 · public part · arts | 50.0 | Setup & sourceDecision Index 0.3 public part, arts area, with the board’s own kit. Amos 1 large, card d67e051. Public part with the board’s kit; area score. Printed in the vendor’s own card table. Scira’s own results on the four Amos 1 Hugging Face cards, each read at the pinned card revision. “As served” rows use Scira’s hosted or packaged server; the cards name no hardware for the score tables. Each value cites its own card. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-13 · Revision: amos-1-large card d67e051c763dc88a1e3662a5003c1e1ea5ffaa74; amos-1-medium card c7f72b601b276f635fb5a9c1901bd96a10a60fe8; amos-1-small card 88713c7be8e5f8122272d276ec3c44ff0528d7e2; amos-1-mini card a59a936bf608dc8fc3ae47ad7715af2153150f3b · Published observations; result reuse terms not specified Result source · Benchmark source |
Evaluator-published · Benchmark Heaven
| Evaluation | Result | Evidence |
|---|---|---|
| Benchmark Heaven JevBench v1.8 · Capability Score | 77.989 | Setup & sourceCapability Score on the rendered board. Amos 1 large (Scira) · Open weights · LLM decoder. Board key amos-1-large; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-large. Rendered board row. The row links the Amos 1 large card and names no commit; the weights are unchanged since 13 October. Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation. Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained Result source · Benchmark source |
| Benchmark Heaven JevBench v1.8 · Intelligence | 68.2 | Setup & sourceIntelligence on the rendered board. Amos 1 large (Scira) · Open weights · LLM decoder. Board key amos-1-large; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-large. Rendered board row. The row links the Amos 1 large card and names no commit; the weights are unchanged since 13 October. Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation. Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained Result source · Benchmark source |
| Benchmark Heaven JevBench v1.8 · Calibration | 87.8 | Setup & sourceCalibration on the rendered board. Amos 1 large (Scira) · Open weights · LLM decoder. Board key amos-1-large; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-large. Rendered board row. The row links the Amos 1 large card and names no commit; the weights are unchanged since 13 October. Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation. Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained Result source · Benchmark source |
| Benchmark Heaven JevBench v1.8 · Cost Score | 30.0 | Setup & sourceCost Score on the rendered board. Amos 1 large (Scira) · Open weights · LLM decoder. Board key amos-1-large; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-large. Rendered board row. The row links the Amos 1 large card and names no commit; the weights are unchanged since 13 October. Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation. Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained Result source · Benchmark source |
| Benchmark Heaven JevBench v1.8 · Modelled cost per 1,000 decisions | 0.21462 USD/1k decisions | Setup & sourceModelled cost per 1,000 decisions on the rendered board. Amos 1 large (Scira) · Open weights · LLM decoder. Board key amos-1-large; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-large. Rendered board row. The row links the Amos 1 large card and names no commit; the weights are unchanged since 13 October. Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation. Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained Result source · Benchmark source |
| Benchmark Heaven JevBench v1.8 · API latency, end to end | 0.53 s | Setup & sourceAPI latency, end to end on the rendered board. Amos 1 large (Scira) · Open weights · LLM decoder. Board key amos-1-large; method tag v1.7. Measured on the Second regular draw (v1.6.43), 2026-10-13; put on the v1.6.1 scale by the draw's anchor offset (see How it works). Source link https://huggingface.co/mukaddamzaid/amos-1-large. Rendered board row. The row links the Amos 1 large card and names no commit; the weights are unchanged since 13 October. Benchmark Heaven's rendered board, read 2 November 2026: 283 ranked systems · 1,500 decisions each (600 for 19 equated API re-runs) · 131 newer ranked rows tagged with their draw · 14 older rows dated separately. Newer rows are measured on dated draws and put on the v1.6.1 scale by the draw's anchor offset, so a row's draw is part of its identity. Distinct from Leanmcp JevBench, the community Decision Index and the stored Benchmark Heaven v1.6.1 feed. Cost is the source's model on a GPU-time basis for self-hosted rows, not a provider tariff. Self-hosted latency is timed on the GPU over loopback and API latency end to end with the network; the source says they are not directly comparable, so they stay in separate datasets. No Examenos recomputation. Checked / reported: 2026-11-08 · Evaluated: 2026-10-13 · Samples: Not published Snapshot: 2026-10-31 · Revision: Board v1.7.48, method v1.8, updated 2026-10-31; page sha256 df9093384c9cdf5854e065f71dd3abd83a127793f22f4ca70268e9981e0a0b17 · Published summary observations; upstream task/result terms retained Result source · Benchmark source |
Official / vendor · Typed Decisions · creator-reported
| Evaluation | Result | Evidence |
|---|---|---|
| Typed Decisions · Creator-reported rows, 5 November · Accuracy | 76.10% | Setup & sourceAccuracy for a creator-reported general zero-shot row on the Typed Decisions board. Exact Amos 1 large row, source label general, zero-shot, open weights; latency on an H100; the row names no revision. Creator-reported board row. Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations Result source · Benchmark source |
| Typed Decisions · Creator-reported rows, 5 November · KL from gold | 0.126 KL | Setup & sourceKL from gold for a creator-reported general zero-shot row on the Typed Decisions board. Exact Amos 1 large row, source label general, zero-shot, open weights; latency on an H100; the row names no revision. Creator-reported board row. Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations Result source · Benchmark source |
| Typed Decisions · Creator-reported rows, 5 November · Brier | 0.069 Brier | Setup & sourceBrier for a creator-reported general zero-shot row on the Typed Decisions board. Exact Amos 1 large row, source label general, zero-shot, open weights; latency on an H100; the row names no revision. Creator-reported board row. Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations Result source · Benchmark source |
| Typed Decisions · Creator-reported rows, 5 November · ECE | 9.7 pp | Setup & sourceECE for a creator-reported general zero-shot row on the Typed Decisions board. Exact Amos 1 large row, source label general, zero-shot, open weights; latency on an H100; the row names no revision. Creator-reported board row. Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations Result source · Benchmark source |
| Typed Decisions · Creator-reported rows, 5 November · Median latency | 357 ms | Setup & sourceMedian latency for a creator-reported general zero-shot row on the Typed Decisions board. Exact Amos 1 large row, source label general, zero-shot, open weights; latency on an H100; the row names no revision. Creator-reported board row. Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations Result source · Benchmark source |
| Typed Decisions · Creator-reported rows, 5 November · Fitted on train · Accuracy | 80.25% | Setup & sourceFitted on train · Accuracy for a creator-reported row in the board’s table of models fitted or fine-tuned on train; not comparable with the zero-shot table. Exact Amos 1 large row with per-question priors fitted on train (one log-prior bias per question and option; no weights changed); Brier and ECE not published. Creator-reported board row. Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations Result source · Benchmark source |
| Typed Decisions · Creator-reported rows, 5 November · Fitted on train · KL from gold | 0.122 KL | Setup & sourceFitted on train · KL from gold for a creator-reported row in the board’s table of models fitted or fine-tuned on train; not comparable with the zero-shot table. Exact Amos 1 large row with per-question priors fitted on train (one log-prior bias per question and option; no weights changed); Brier and ECE not published. Creator-reported board row. Amos rows on the Typed Decisions card at revision f5affa5 (5 November 2026), which lists them under rows reported by the model’s creator (discussion #31). Separate from the creator rows pinned at 94993b8 and from Scira’s own card values, which round the same runs. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-11-05T13:04:11.000Z · Revision: f5affa5f05859bd8486bc15e1e352a04f67994cf · Apache-2.0 dataset/card; creator-reported observations Result source · Benchmark source |
Evaluator-published · Community Decision Index
| Evaluation | Result | Evidence |
|---|---|---|
| HF text 0.4.1 · Full score | 64.81 | Setup & sourceHF text 0.4.1 · Full score Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Public index | 64.94 | Setup & sourceHF text 0.4 · Public index Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Private same-skill component | 62.69 | Setup & sourceHF text 0.4 · Private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4.1 · Private new-domain component | 53.31 | Setup & sourceHF text 0.4.1 · Private new-domain component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Raw public index | 73.54 | Setup & sourceHF text 0.4 · Raw public index Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Public breadth index | 64.15 | Setup & sourceHF text 0.4 · Public breadth index Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Equated private same-skill | 62.37 | Setup & sourceHF text 0.4 · Equated private same-skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4.1 · Equated private new domains | 68.48 | Setup & sourceHF text 0.4.1 · Equated private new domains Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Full contribution · public | 0.00 | Setup & sourceHF text 0.4 · Full contribution · public Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Full contribution · same_skills | 37.42 | Setup & sourceHF text 0.4 · Full contribution · same_skills Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4.1 · Full contribution · new_domains | 27.39 | Setup & sourceHF text 0.4.1 · Full contribution · new_domains Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published weighted contribution; not a distinct benchmark execution. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Community Decision Index 0.4.1: expanded/refreshed harder private new-domain tasks and equating; Full remains 60% private same-skill +40% private new domains, zero public contribution. This population revision is not a model upgrade/regression. Public/native/calibration data remain on their unchanged 0.4 definitions. Source publication/generation is not a model run date. Public submitted runs plus evaluator private measurements are Evaluator-published, not wholly independently executed. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-22T05:40:06Z · Revision: d9962113a1acc95c2d7522c2babfb989afbeb034; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Knowledge · raw | 61.67 | Setup & sourceHF text 0.4 · Knowledge · raw Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6167, "skill": 0.5236, "combined": 0.4462, "coverage": 1, "pending": 0, "n": 11} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 11 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Knowledge · skill | 52.36 | Setup & sourceHF text 0.4 · Knowledge · skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6167, "skill": 0.5236, "combined": 0.4462, "coverage": 1, "pending": 0, "n": 11} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 11 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Knowledge · combined | 44.62 | Setup & sourceHF text 0.4 · Knowledge · combined Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "knowledge", "raw": 0.6167, "skill": 0.5236, "combined": 0.4462, "coverage": 1, "pending": 0, "n": 11} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 11 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Language · raw | 80.39 | Setup & sourceHF text 0.4 · Language · raw Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.8039, "skill": 0.7244, "combined": 0.6592, "coverage": 1, "pending": 0, "n": 9} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 9 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Language · skill | 72.44 | Setup & sourceHF text 0.4 · Language · skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.8039, "skill": 0.7244, "combined": 0.6592, "coverage": 1, "pending": 0, "n": 9} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 9 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Language · combined | 65.92 | Setup & sourceHF text 0.4 · Language · combined Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "language", "raw": 0.8039, "skill": 0.7244, "combined": 0.6592, "coverage": 1, "pending": 0, "n": 9} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 9 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Retrieval · raw | 75.53 | Setup & sourceHF text 0.4 · Retrieval · raw Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7553, "skill": 0.6615, "combined": 0.6695, "coverage": 1, "pending": 0, "n": 6} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Retrieval · skill | 66.15 | Setup & sourceHF text 0.4 · Retrieval · skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7553, "skill": 0.6615, "combined": 0.6695, "coverage": 1, "pending": 0, "n": 6} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Retrieval · combined | 66.95 | Setup & sourceHF text 0.4 · Retrieval · combined Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "retrieval", "raw": 0.7553, "skill": 0.6615, "combined": 0.6695, "coverage": 1, "pending": 0, "n": 6} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Tools · raw | 82.09 | Setup & sourceHF text 0.4 · Tools · raw Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.8209, "skill": 0.7923, "combined": 0.7957, "coverage": 1, "pending": 0, "n": 5} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Tools · skill | 79.23 | Setup & sourceHF text 0.4 · Tools · skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.8209, "skill": 0.7923, "combined": 0.7957, "coverage": 1, "pending": 0, "n": 5} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Tools · combined | 79.57 | Setup & sourceHF text 0.4 · Tools · combined Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "tools", "raw": 0.8209, "skill": 0.7923, "combined": 0.7957, "coverage": 1, "pending": 0, "n": 5} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Arts · raw | 66.92 | Setup & sourceHF text 0.4 · Arts · raw Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.6692, "skill": 0.495, "combined": 0.5331, "coverage": 1, "pending": 0, "n": 6} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Arts · skill | 49.50 | Setup & sourceHF text 0.4 · Arts · skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.6692, "skill": 0.495, "combined": 0.5331, "coverage": 1, "pending": 0, "n": 6} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Arts · combined | 53.31 | Setup & sourceHF text 0.4 · Arts · combined Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Count is contributing benchmarks, not requests. Source area context: {"id": "arts", "raw": 0.6692, "skill": 0.495, "combined": 0.5331, "coverage": 1, "pending": 0, "n": 6} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 6 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · ECE | 3.17 pp | Setup & sourceHF text calibration 0.4 · ECE Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 34411 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Brier | 0.3126 Brier | Setup & sourceHF text calibration 0.4 · Brier Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 34411 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Sample accuracy | 76.64% | Setup & sourceHF text calibration 0.4 · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 34411 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Mean chosen confidence | 75.93% | Setup & sourceHF text calibration 0.4 · Mean chosen confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 34411 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Wrong among answers at ≥95% confidence | 0.07% | Setup & sourceHF text calibration 0.4 · Wrong among answers at ≥95% confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too. Contributing benchmarks 31. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 34411 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Knowledge · ECE | 7.26 pp | Setup & sourceHF text calibration 0.4 · Knowledge · ECE Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Knowledge · Sample accuracy | 69.15% | Setup & sourceHF text calibration 0.4 · Knowledge · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Knowledge · Mean chosen confidence | 73.00% | Setup & sourceHF text calibration 0.4 · Knowledge · Mean chosen confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Language · ECE | 8.27 pp | Setup & sourceHF text calibration 0.4 · Language · ECE Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Language · Sample accuracy | 89.97% | Setup & sourceHF text calibration 0.4 · Language · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Language · Mean chosen confidence | 81.70% | Setup & sourceHF text calibration 0.4 · Language · Mean chosen confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Retrieval · ECE | 13.76 pp | Setup & sourceHF text calibration 0.4 · Retrieval · ECE Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Retrieval · Sample accuracy | 66.56% | Setup & sourceHF text calibration 0.4 · Retrieval · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Retrieval · Mean chosen confidence | 76.91% | Setup & sourceHF text calibration 0.4 · Retrieval · Mean chosen confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Tools · ECE | 4.17 pp | Setup & sourceHF text calibration 0.4 · Tools · ECE Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Tools · Sample accuracy | 95.08% | Setup & sourceHF text calibration 0.4 · Tools · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Tools · Mean chosen confidence | 91.92% | Setup & sourceHF text calibration 0.4 · Tools · Mean chosen confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Arts · ECE | 3.95 pp | Setup & sourceHF text calibration 0.4 · Arts · ECE Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Arts · Sample accuracy | 64.65% | Setup & sourceHF text calibration 0.4 · Arts · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Arts · Mean chosen confidence | 61.06% | Setup & sourceHF text calibration 0.4 · Arts · Mean chosen confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Area-specific item counts unpublished; do not use whole-sample count. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0–0.1 · Sample weight | 0.20% | Setup & sourceHF text calibration 0.4 · Confidence bin 0–0.1 · Sample weight Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0–0.1 · Mean confidence | 8.47% | Setup & sourceHF text calibration 0.4 · Confidence bin 0–0.1 · Mean confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0–0.1 · Sample accuracy | 13.54% | Setup & sourceHF text calibration 0.4 · Confidence bin 0–0.1 · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.1–0.2 · Sample weight | 2.01% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.1–0.2 · Sample weight Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.1. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.1–0.2 · Mean confidence | 15.17% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.1–0.2 · Mean confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.1. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.1–0.2 · Sample accuracy | 13.72% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.1–0.2 · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.1. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.2–0.3 · Sample weight | 2.46% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.2–0.3 · Sample weight Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.2. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.2–0.3 · Mean confidence | 25.18% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.2–0.3 · Mean confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.2. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.2–0.3 · Sample accuracy | 30.33% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.2–0.3 · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.2. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.3–0.4 · Sample weight | 3.59% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.3–0.4 · Sample weight Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.3. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.3–0.4 · Mean confidence | 35.49% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.3–0.4 · Mean confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.3. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.3–0.4 · Sample accuracy | 28.76% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.3–0.4 · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.3. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.4–0.5 · Sample weight | 5.12% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.4–0.5 · Sample weight Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.4. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.4–0.5 · Mean confidence | 45.10% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.4–0.5 · Mean confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.4. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.4–0.5 · Sample accuracy | 37.45% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.4–0.5 · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.4. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.5–0.6 · Sample weight | 8.76% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.5–0.6 · Sample weight Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.5. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.5–0.6 · Mean confidence | 54.89% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.5–0.6 · Mean confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.5. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.5–0.6 · Sample accuracy | 51.91% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.5–0.6 · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.5. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.6–0.7 · Sample weight | 9.05% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.6–0.7 · Sample weight Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.6. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.6–0.7 · Mean confidence | 65.13% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.6–0.7 · Mean confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.6. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.6–0.7 · Sample accuracy | 61.71% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.6–0.7 · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.6. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.7–0.8 · Sample weight | 10.70% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.7–0.8 · Sample weight Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.7. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.7–0.8 · Mean confidence | 75.44% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.7–0.8 · Mean confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.7. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.7–0.8 · Sample accuracy | 76.04% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.7–0.8 · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.7. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.8–0.9 · Sample weight | 23.01% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.8–0.9 · Sample weight Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.8. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.8–0.9 · Mean confidence | 85.58% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.8–0.9 · Mean confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.8. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.8–0.9 · Sample accuracy | 86.99% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.8–0.9 · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.8. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.9–1.0 · Sample weight | 35.10% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.9–1.0 · Sample weight Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.9. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.9–1.0 · Mean confidence | 93.86% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.9–1.0 · Mean confidence Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.9. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text calibration 0.4 · Confidence bin 0.9–1.0 · Sample accuracy | 97.90% | Setup & sourceHF text calibration 0.4 · Confidence bin 0.9–1.0 · Sample accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Reliability bin; source weighted fraction, not sample count. Lower boundary 0.9. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Local median latency | 147.90 ms | Setup & sourceHF text 0.4 · Local median latency Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Latency is measured on its own, after scoring: every entrant runs the same 750 rows (stratified by benchmark and prompt length across the 0.2 scope, plus 10 untimed warm-up rows) as one process sending one request at a time on 1 x NVIDIA RTX PRO 6000, using the fastest inference path the model or its code supports. The figure is the wall time of one row, request in to all answers out, measured the same way for in-process and HTTP engines; board medians and p95s come from this run, not from the scoring runs. Entrant-specific serving/hardware caveats in configuration take precedence; HTTPS Jev is not isolated local inference. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Local p95 latency | 1365.80 ms | Setup & sourceHF text 0.4 · Local p95 latency Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Latency is measured on its own, after scoring: every entrant runs the same 750 rows (stratified by benchmark and prompt length across the 0.2 scope, plus 10 untimed warm-up rows) as one process sending one request at a time on 1 x NVIDIA RTX PRO 6000, using the fastest inference path the model or its code supports. The figure is the wall time of one row, request in to all answers out, measured the same way for in-process and HTTP engines; board medians and p95s come from this run, not from the scoring runs. Entrant-specific serving/hardware caveats in configuration take precedence; HTTPS Jev is not isolated local inference. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Local mean latency | 460.70 ms | Setup & sourceHF text 0.4 · Local mean latency Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Latency is measured on its own, after scoring: every entrant runs the same 750 rows (stratified by benchmark and prompt length across the 0.2 scope, plus 10 untimed warm-up rows) as one process sending one request at a time on 1 x NVIDIA RTX PRO 6000, using the fastest inference path the model or its code supports. The figure is the wall time of one row, request in to all answers out, measured the same way for in-process and HTTP engines; board medians and p95s come from this run, not from the scoring runs. Entrant-specific serving/hardware caveats in configuration take precedence; HTTPS Jev is not isolated local inference. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BFCL · case exact accuracy | 97.93% | Setup & sourceHF text 0.4 · BFCL · case exact accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9793, "jev": null, "metric": "case exact accuracy", "requests": 1694, "answered": 1694, "unsupported": 0, "errors": 0, "median_ms": 254.6}. Benchmark contract: {"id": 1, "dataset": "BFCL", "short": "BFCL", "metric": "case exact accuracy", "explainer": "Says yes or no for each supplied function: call it for this request? All must be right.", "baseline": {"value": 0.2592, "label": "Random whole-case guesses"}, "lower": false, "range": null, "cases": 1694, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2305.15334"}, {"kind": "code", "url": "https://github.com/gorilla-llm/gorilla"}], "gold": 1.2, "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1694 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BFCL · median_ms | 254.60 ms | Setup & sourceHF text 0.4 · BFCL · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BFCL · Raw index component | 97.93 | Setup & sourceHF text 0.4 · BFCL · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9793, "skill": 0.9721, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9874, "private_eq": 0.99} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1694 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BFCL · Chance-corrected public skill | 97.21 | Setup & sourceHF text 0.4 · BFCL · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9793, "skill": 0.9721, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9874, "private_eq": 0.99} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1694 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BFCL · Combined public/private component | 98.74 | Setup & sourceHF text 0.4 · BFCL · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9793, "skill": 0.9721, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9874, "private_eq": 0.99} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BFCL · Equated private same-skill component | 99.00 | Setup & sourceHF text 0.4 · BFCL · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.9793, "skill": 0.9721, "coverage": 1, "random": 0.2592, "tracks": [], "combined": 0.9874, "private_eq": 0.99} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ToolRet · nDCG@10 | 68.94% | Setup & sourceHF text 0.4 · ToolRet · nDCG@10 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6894, "jev": null, "metric": "nDCG@10", "requests": 685, "answered": 685, "unsupported": 0, "errors": 0, "median_ms": 2379.3}. Benchmark contract: {"id": 2, "dataset": "ToolRet", "short": "ToolRet", "metric": "nDCG@10", "explainer": "Rates how useful each of 32 search-hit tools is for a request; ranked by nDCG@10, on requests where at least one of the 32 is relevant.", "baseline": {"value": 0.1341, "label": "Random ranking, answerable queries", "previous": 0.0937}, "lower": false, "range": null, "cases": 685, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2503.01763"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/mangopy/ToolRet-Queries"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 685 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ToolRet · median_ms | 2379.30 ms | Setup & sourceHF text 0.4 · ToolRet · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ToolRet · Raw index component | 68.94 | Setup & sourceHF text 0.4 · ToolRet · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.6894, "skill": 0.6413, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7104, "private_eq": 0.7512} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 685 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ToolRet · Chance-corrected public skill | 64.13 | Setup & sourceHF text 0.4 · ToolRet · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.6894, "skill": 0.6413, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7104, "private_eq": 0.7512} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 685 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ToolRet · Combined public/private component | 71.04 | Setup & sourceHF text 0.4 · ToolRet · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.6894, "skill": 0.6413, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7104, "private_eq": 0.7512} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ToolRet · Equated private same-skill component | 75.12 | Setup & sourceHF text 0.4 · ToolRet · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.6894, "skill": 0.6413, "coverage": 1, "random": 0.1341, "tracks": [], "combined": 0.7104, "private_eq": 0.7512} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · API-Bank · accuracy | 86.02% | Setup & sourceHF text 0.4 · API-Bank · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8602, "jev": null, "metric": "accuracy", "requests": 508, "answered": 508, "unsupported": 0, "errors": 0, "median_ms": 3127.8}. Benchmark contract: {"id": 3, "dataset": "API-Bank", "short": "API-Bank", "metric": "accuracy", "explainer": "Reads an assistant conversation and picks the next API to call from a catalog of 53.", "baseline": {"value": 0.0189, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 508, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2304.08244"}, {"kind": "code", "url": "https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/api-bank"}], "gold": 1.2, "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 508 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · API-Bank · median_ms | 3127.80 ms | Setup & sourceHF text 0.4 · API-Bank · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · API-Bank · Raw index component | 86.02 | Setup & sourceHF text 0.4 · API-Bank · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8602, "skill": 0.8575, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.9013, "private_eq": 0.9033} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 508 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · API-Bank · Chance-corrected public skill | 85.75 | Setup & sourceHF text 0.4 · API-Bank · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8602, "skill": 0.8575, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.9013, "private_eq": 0.9033} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 508 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · API-Bank · Combined public/private component | 90.13 | Setup & sourceHF text 0.4 · API-Bank · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8602, "skill": 0.8575, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.9013, "private_eq": 0.9033} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · API-Bank · Equated private same-skill component | 90.33 | Setup & sourceHF text 0.4 · API-Bank · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8602, "skill": 0.8575, "coverage": 1, "random": 0.0189, "tracks": [], "combined": 0.9013, "private_eq": 0.9033} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BANKING77 · macro-F1 | 88.91% | Setup & sourceHF text 0.4 · BANKING77 · macro-F1 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8891, "jev": null, "metric": "macro-F1", "requests": 3080, "answered": 3080, "unsupported": 0, "errors": 0, "median_ms": 253}. Benchmark contract: {"id": 4, "dataset": "BANKING77", "short": "BANKING77", "metric": "macro-F1", "explainer": "Picks the intent of a bank customer's message, e.g. \"How do I locate my card?\", from 77.", "baseline": {"value": 0.0127, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 3080, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2003.04807"}, {"kind": "dataset", "url": "https://github.com/PolyAI-LDN/task-specific-datasets"}], "gold": 1.2, "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3080 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BANKING77 · median_ms | 253.00 ms | Setup & sourceHF text 0.4 · BANKING77 · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BANKING77 · Raw index component | 88.91 | Setup & sourceHF text 0.4 · BANKING77 · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8891, "skill": 0.8877, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.8601, "private_eq": 0.8625} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3080 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BANKING77 · Chance-corrected public skill | 88.77 | Setup & sourceHF text 0.4 · BANKING77 · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8891, "skill": 0.8877, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.8601, "private_eq": 0.8625} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3080 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BANKING77 · Combined public/private component | 86.01 | Setup & sourceHF text 0.4 · BANKING77 · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8891, "skill": 0.8877, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.8601, "private_eq": 0.8625} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BANKING77 · Equated private same-skill component | 86.25 | Setup & sourceHF text 0.4 · BANKING77 · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8891, "skill": 0.8877, "coverage": 1, "random": 0.0127, "tracks": [], "combined": 0.8601, "private_eq": 0.8625} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLINC150+OOS · macro-F1 | 94.84% | Setup & sourceHF text 0.4 · CLINC150+OOS · macro-F1 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9484, "jev": null, "metric": "macro-F1", "requests": 5500, "answered": 5500, "unsupported": 0, "errors": 0, "median_ms": 417.2}. Benchmark contract: {"id": 5, "dataset": "CLINC150+OOS", "short": "CLINC150", "metric": "macro-F1", "explainer": "Reads a short assistant request and picks its intent from 150, or \"out of scope\" when none fits.", "baseline": {"value": 0.006, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 5500, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/clinc/oos-eval"}], "gold": 1.2, "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5500 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLINC150+OOS · median_ms | 417.20 ms | Setup & sourceHF text 0.4 · CLINC150+OOS · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLINC150+OOS · Raw index component | 94.84 | Setup & sourceHF text 0.4 · CLINC150+OOS · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9484, "skill": 0.9481, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.7697, "private_eq": 0.7727} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5500 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLINC150+OOS · Chance-corrected public skill | 94.81 | Setup & sourceHF text 0.4 · CLINC150+OOS · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9484, "skill": 0.9481, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.7697, "private_eq": 0.7727} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5500 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLINC150+OOS · Combined public/private component | 76.97 | Setup & sourceHF text 0.4 · CLINC150+OOS · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9484, "skill": 0.9481, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.7697, "private_eq": 0.7727} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLINC150+OOS · Equated private same-skill component | 77.27 | Setup & sourceHF text 0.4 · CLINC150+OOS · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9484, "skill": 0.9481, "coverage": 1, "random": 0.006, "tracks": [], "combined": 0.7697, "private_eq": 0.7727} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · RouterBench · selected quality (quality objective) | 79.53% | Setup & sourceHF text 0.4 · RouterBench · selected quality (quality objective) Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7953, "jev": null, "metric": "selected quality (quality objective)", "requests": 10000, "answered": 10000, "unsupported": 0, "errors": 0, "median_ms": 723}. Benchmark contract: {"id": 6, "dataset": "RouterBench", "short": "RouterBench", "metric": "selected quality (quality objective)", "explainer": "Picks which of 11 AI models should answer a prompt; scored by its real answer.", "baseline": {"value": 0.5735, "label": "Random model selection"}, "lower": false, "range": null, "cases": 5001, "group": "tools", "in_suite": true, "interactive": false, "index_note": "Not in the index: the prompt reveals that GPT-4 is the best route, so the top models all pick GPT-4 and score within a fraction of a point of each other. Scores stay visible.", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2403.12031"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/withmartian/routerbench"}]} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 10000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · RouterBench · median_ms | 723.00 ms | Setup & sourceHF text 0.4 · RouterBench · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · RouterBench · Raw index component | 79.53 | Setup & sourceHF text 0.4 · RouterBench · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native selected quality (quality objective). Source normalization: {"raw": 0.7953, "skill": 0.52, "coverage": 1, "random": 0.5735, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 10000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · RouterBench · Chance-corrected public skill | 52.00 | Setup & sourceHF text 0.4 · RouterBench · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native selected quality (quality objective). Source normalization: {"raw": 0.7953, "skill": 0.52, "coverage": 1, "random": 0.5735, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 10000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Home appliance simulator · case exact accuracy | 81.82% | Setup & sourceHF text 0.4 · Home appliance simulator · case exact accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8182, "jev": null, "metric": "case exact accuracy", "requests": 88, "answered": 88, "unsupported": 0, "errors": 0, "median_ms": 737.8}. Benchmark contract: {"id": 9, "dataset": "Home appliance simulator", "short": "Home appliances", "metric": "case exact accuracy", "explainer": "Given a simulated home and a command, answers ~18 questions on handling it; all must be right. Duplicate and dev-set rows removed.", "baseline": {"value": 0, "label": "Random whole-case guesses"}, "lower": false, "range": null, "cases": 88, "group": "tools", "in_suite": true, "interactive": false, "references": [{"kind": "code", "url": "https://github.com/apolinario/decision-index/blob/main/decision_index/suite/build/home_appliance.py"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 88 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Home appliance simulator · median_ms | 737.80 ms | Setup & sourceHF text 0.4 · Home appliance simulator · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Home appliance simulator · Raw index component | 81.82 | Setup & sourceHF text 0.4 · Home appliance simulator · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.8182, "skill": 0.8182, "coverage": 1, "random": 0, "tracks": [], "combined": 0.7431, "private_eq": 0.745} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 88 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Home appliance simulator · Chance-corrected public skill | 81.82 | Setup & sourceHF text 0.4 · Home appliance simulator · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.8182, "skill": 0.8182, "coverage": 1, "random": 0, "tracks": [], "combined": 0.7431, "private_eq": 0.745} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 88 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Home appliance simulator · Combined public/private component | 74.31 | Setup & sourceHF text 0.4 · Home appliance simulator · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.8182, "skill": 0.8182, "coverage": 1, "random": 0, "tracks": [], "combined": 0.7431, "private_eq": 0.745} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Home appliance simulator · Equated private same-skill component | 74.50 | Setup & sourceHF text 0.4 · Home appliance simulator · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.8182, "skill": 0.8182, "coverage": 1, "random": 0, "tracks": [], "combined": 0.7431, "private_eq": 0.745} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · SGD/SGD-X · macro-F1 | 11.83% | Setup & sourceHF text 0.4 · SGD/SGD-X · macro-F1 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.1183, "jev": null, "metric": "macro-F1", "requests": 2500, "answered": 2500, "unsupported": 0, "errors": 0, "median_ms": 432.5}. Benchmark contract: {"id": 10, "dataset": "SGD/SGD-X", "short": "SGD", "metric": "macro-F1", "explainer": "Reads a service dialogue so far and picks the intent the user is pursuing now, or NONE.", "baseline": {"value": 0.399, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 2500, "group": "retrieval", "in_suite": true, "interactive": false, "index_note": "Not in the index: build bug, pending re-run. The builder gave every SGD row the whole dialogue, including the turns after the one being classified. Scores stay visible; SGD returns to the index once re-run with the fixed builder.", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/1909.05855"}, {"kind": "dataset", "url": "https://github.com/google-research-datasets/dstc8-schema-guided-dialogue"}]} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2500 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · SGD/SGD-X · median_ms | 432.50 ms | Setup & sourceHF text 0.4 · SGD/SGD-X · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · SGD/SGD-X · Raw index component | 11.83 | Setup & sourceHF text 0.4 · SGD/SGD-X · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.1183, "skill": 0, "coverage": 1, "random": 0.399, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2500 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · SGD/SGD-X · Chance-corrected public skill | 0.00 | Setup & sourceHF text 0.4 · SGD/SGD-X · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.1183, "skill": 0, "coverage": 1, "random": 0.399, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2500 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ContractNLI · macro-F1 | 84.71% | Setup & sourceHF text 0.4 · ContractNLI · macro-F1 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8471, "jev": null, "metric": "macro-F1", "requests": 123, "answered": 123, "unsupported": 0, "errors": 0, "median_ms": 887.8}. Benchmark contract: {"id": 11, "dataset": "ContractNLI", "short": "ContractNLI", "metric": "macro-F1", "explainer": "Says whether a full NDA supports, contradicts or omits each of 17 fixed statements.", "baseline": {"value": 0.3085, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 123, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2021.findings-emnlp.164"}, {"kind": "dataset", "url": "https://stanfordnlp.github.io/contract-nli/"}]} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 123 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ContractNLI · median_ms | 887.80 ms | Setup & sourceHF text 0.4 · ContractNLI · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ContractNLI · Raw index component | 84.71 | Setup & sourceHF text 0.4 · ContractNLI · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8471, "skill": 0.7789, "coverage": 1, "random": 0.3085, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 123 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ContractNLI · Chance-corrected public skill | 77.89 | Setup & sourceHF text 0.4 · ContractNLI · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8471, "skill": 0.7789, "coverage": 1, "random": 0.3085, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 123 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ANLI · macro-F1 | 77.77% | Setup & sourceHF text 0.4 · ANLI · macro-F1 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7777, "jev": null, "metric": "macro-F1", "requests": 3200, "answered": 3200, "unsupported": 0, "errors": 0, "median_ms": 119.6}. Benchmark contract: {"id": 12, "dataset": "ANLI", "short": "ANLI", "metric": "macro-F1", "explainer": "Decides whether a claim follows from, contradicts, or is neutral to a premise written to fool AI models.", "baseline": {"value": 0.3324, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 3200, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/1910.14599"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/facebook/anli"}], "gold": 1.2, "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3200 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ANLI · median_ms | 119.60 ms | Setup & sourceHF text 0.4 · ANLI · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ANLI · Raw index component | 77.77 | Setup & sourceHF text 0.4 · ANLI · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7777, "skill": 0.667, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5871, "private_eq": 0.7268} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3200 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ANLI · Chance-corrected public skill | 66.70 | Setup & sourceHF text 0.4 · ANLI · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7777, "skill": 0.667, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5871, "private_eq": 0.7268} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3200 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ANLI · Combined public/private component | 58.71 | Setup & sourceHF text 0.4 · ANLI · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7777, "skill": 0.667, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5871, "private_eq": 0.7268} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ANLI · Equated private same-skill component | 72.68 | Setup & sourceHF text 0.4 · ANLI · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7777, "skill": 0.667, "coverage": 1, "random": 0.3324, "tracks": [], "combined": 0.5871, "private_eq": 0.7268} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BPoMP · accuracy | 96.70% | Setup & sourceHF text 0.4 · BPoMP · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.967, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 119.9}. Benchmark contract: {"id": 20, "dataset": "BPoMP", "short": "BPoMP", "metric": "accuracy", "explainer": "Given two versions of a limerick, one with its rhyme, meter or sense broken, picks the original.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 811, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://zenodo.org/records/7299879"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BPoMP · median_ms | 119.90 ms | Setup & sourceHF text 0.4 · BPoMP · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BPoMP · Raw index component | 96.86 | Setup & sourceHF text 0.4 · BPoMP · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9686, "skill": 0.9372, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9432, "private_eq": 0.9714} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BPoMP · Chance-corrected public skill | 93.72 | Setup & sourceHF text 0.4 · BPoMP · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9686, "skill": 0.9372, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9432, "private_eq": 0.9714} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BPoMP · Combined public/private component | 94.32 | Setup & sourceHF text 0.4 · BPoMP · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9686, "skill": 0.9372, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9432, "private_eq": 0.9714} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BPoMP · Equated private same-skill component | 97.14 | Setup & sourceHF text 0.4 · BPoMP · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9686, "skill": 0.9372, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.9432, "private_eq": 0.9714} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Humicroedit · accuracy | 66.51% | Setup & sourceHF text 0.4 · Humicroedit · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6651, "jev": null, "metric": "accuracy", "requests": 2628, "answered": 2628, "unsupported": 0, "errors": 0, "median_ms": 119.4}. Benchmark contract: {"id": 21, "dataset": "Humicroedit", "short": "Humicroedit", "metric": "accuracy", "explainer": "Given two one-word edits of a news headline, picks the one people rated funnier.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2628, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/1906.00274"}, {"kind": "dataset", "url": "https://cs.rochester.edu/u/nhossain/humicroedit.html"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2628 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Humicroedit · median_ms | 119.40 ms | Setup & sourceHF text 0.4 · Humicroedit · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Humicroedit · Raw index component | 66.51 | Setup & sourceHF text 0.4 · Humicroedit · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6651, "skill": 0.3303, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.2968, "private_eq": 0.6519} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2628 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Humicroedit · Chance-corrected public skill | 33.03 | Setup & sourceHF text 0.4 · Humicroedit · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6651, "skill": 0.3303, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.2968, "private_eq": 0.6519} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2628 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Humicroedit · Combined public/private component | 29.68 | Setup & sourceHF text 0.4 · Humicroedit · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6651, "skill": 0.3303, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.2968, "private_eq": 0.6519} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Humicroedit · Equated private same-skill component | 65.19 | Setup & sourceHF text 0.4 · Humicroedit · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6651, "skill": 0.3303, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.2968, "private_eq": 0.6519} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · POP909-CL · accuracy | 57.00% | Setup & sourceHF text 0.4 · POP909-CL · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.57, "jev": null, "metric": "accuracy", "requests": 2000, "answered": 2000, "unsupported": 0, "errors": 0, "median_ms": 2640.1}. Benchmark contract: {"id": 22, "dataset": "POP909-CL", "short": "POP909", "metric": "accuracy", "explainer": "Given the notes around a beat in a pop song (not audio), picks the chord playing from 129.", "baseline": {"value": 0.0078, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2000, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2510.06528"}, {"kind": "dataset", "url": "https://github.com/AndyWeasley2004/POP909-CL-Dataset"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · POP909-CL · median_ms | 2640.10 ms | Setup & sourceHF text 0.4 · POP909-CL · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · POP909-CL · Raw index component | 57.04 | Setup & sourceHF text 0.4 · POP909-CL · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.5704, "skill": 0.567, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5221, "private_eq": 0.5302} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · POP909-CL · Chance-corrected public skill | 56.70 | Setup & sourceHF text 0.4 · POP909-CL · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.5704, "skill": 0.567, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5221, "private_eq": 0.5302} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · POP909-CL · Combined public/private component | 52.21 | Setup & sourceHF text 0.4 · POP909-CL · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.5704, "skill": 0.567, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5221, "private_eq": 0.5302} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · POP909-CL · Equated private same-skill component | 53.02 | Setup & sourceHF text 0.4 · POP909-CL · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.5704, "skill": 0.567, "coverage": 1, "random": 0.0078, "tracks": [], "combined": 0.5221, "private_eq": 0.5302} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · cfcolor · accuracy | 64.98% | Setup & sourceHF text 0.4 · cfcolor · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6498, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 197.9}. Benchmark contract: {"id": 23, "dataset": "cfcolor", "short": "cfcolor", "metric": "accuracy", "explainer": "Given a person's earlier ratings of color palettes, predicts which of two new palettes they rated higher.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 5000, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://www.dgp.toronto.edu/~donovan/cfcolor/"}]} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · cfcolor · median_ms | 197.90 ms | Setup & sourceHF text 0.4 · cfcolor · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · cfcolor · Raw index component | 65.67 | Setup & sourceHF text 0.4 · cfcolor · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6567, "skill": 0.3134, "coverage": 1, "random": 0.5, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · cfcolor · Chance-corrected public skill | 31.34 | Setup & sourceHF text 0.4 · cfcolor · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6567, "skill": 0.3134, "coverage": 1, "random": 0.5, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MMLU · accuracy | 87.71% | Setup & sourceHF text 0.4 · MMLU · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8771, "jev": null, "metric": "accuracy", "requests": 14033, "answered": 14033, "unsupported": 0, "errors": 0, "median_ms": 119.5}. Benchmark contract: {"id": 24, "dataset": "MMLU", "short": "MMLU", "metric": "accuracy", "explainer": "Four-option questions across 57 school and professional subjects, from law to algebra.", "baseline": {"value": 0.25, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 14042, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/cais/mmlu"}]} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 14033 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MMLU · median_ms | 119.50 ms | Setup & sourceHF text 0.4 · MMLU · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MMLU · Raw index component | 87.71 | Setup & sourceHF text 0.4 · MMLU · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8771, "skill": 0.8361, "coverage": 1, "random": 0.25, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 14033 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MMLU · Chance-corrected public skill | 83.61 | Setup & sourceHF text 0.4 · MMLU · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8771, "skill": 0.8361, "coverage": 1, "random": 0.25, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 14033 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GPQA Diamond · accuracy | 56.12% | Setup & sourceHF text 0.4 · GPQA Diamond · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.5612, "jev": null, "metric": "accuracy", "requests": 196, "answered": 196, "unsupported": 0, "errors": 0, "median_ms": 121.6}. Benchmark contract: {"id": 25, "dataset": "GPQA Diamond", "short": "GPQA Diamond", "metric": "accuracy", "explainer": "198 hard four-option science questions written by PhD experts in physics, chemistry and biology.", "baseline": {"value": 0.25, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 198, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/idavidrein/gpqa"}], "gold": 1.2, "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 196 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GPQA Diamond · median_ms | 121.60 ms | Setup & sourceHF text 0.4 · GPQA Diamond · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GPQA Diamond · Raw index component | 56.12 | Setup & sourceHF text 0.4 · GPQA Diamond · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.5612, "skill": 0.415, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.4721, "combined": 0.289} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 196 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GPQA Diamond · Chance-corrected public skill | 41.50 | Setup & sourceHF text 0.4 · GPQA Diamond · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.5612, "skill": 0.415, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.4721, "combined": 0.289} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 196 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GPQA Diamond · Combined public/private component | 28.90 | Setup & sourceHF text 0.4 · GPQA Diamond · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.5612, "skill": 0.415, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.4721, "combined": 0.289} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GPQA Diamond · Equated private same-skill component | 47.21 | Setup & sourceHF text 0.4 · GPQA Diamond · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.5612, "skill": 0.415, "coverage": 1, "random": 0.25, "tracks": [], "private_eq": 0.4721, "combined": 0.289} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ARC-Easy · accuracy | 99.07% | Setup & sourceHF text 0.4 · ARC-Easy · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9907, "jev": null, "metric": "accuracy", "requests": 2376, "answered": 2376, "unsupported": 0, "errors": 0, "median_ms": 118.9}. Benchmark contract: {"id": 26, "dataset": "ARC-Easy", "short": "ARC-Easy", "metric": "accuracy", "explainer": "Grade-school science multiple choice, the easier set.", "baseline": {"value": 0.2502, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2376, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/allenai/ai2_arc"}]} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2376 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ARC-Easy · median_ms | 118.90 ms | Setup & sourceHF text 0.4 · ARC-Easy · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ARC-Easy · Raw index component | 99.07 | Setup & sourceHF text 0.4 · ARC-Easy · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9907, "skill": 0.9876, "coverage": 1, "random": 0.2502, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2376 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ARC-Easy · Chance-corrected public skill | 98.76 | Setup & sourceHF text 0.4 · ARC-Easy · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9907, "skill": 0.9876, "coverage": 1, "random": 0.2502, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2376 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ARC-Challenge · accuracy | 97.44% | Setup & sourceHF text 0.4 · ARC-Challenge · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9744, "jev": null, "metric": "accuracy", "requests": 1172, "answered": 1172, "unsupported": 0, "errors": 0, "median_ms": 118.9}. Benchmark contract: {"id": 27, "dataset": "ARC-Challenge", "short": "ARC-Challenge", "metric": "accuracy", "explainer": "Grade-school science multiple choice, the harder set that simple retrieval gets wrong.", "baseline": {"value": 0.2502, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 1172, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/allenai/ai2_arc"}]} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1172 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ARC-Challenge · median_ms | 118.90 ms | Setup & sourceHF text 0.4 · ARC-Challenge · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ARC-Challenge · Raw index component | 97.44 | Setup & sourceHF text 0.4 · ARC-Challenge · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9744, "skill": 0.9659, "coverage": 1, "random": 0.2502, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1172 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ARC-Challenge · Chance-corrected public skill | 96.59 | Setup & sourceHF text 0.4 · ARC-Challenge · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9744, "skill": 0.9659, "coverage": 1, "random": 0.2502, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1172 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · WinoGrande · accuracy | 93.53% | Setup & sourceHF text 0.4 · WinoGrande · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9353, "jev": null, "metric": "accuracy", "requests": 1267, "answered": 1267, "unsupported": 0, "errors": 0, "median_ms": 118.9}. Benchmark contract: {"id": 28, "dataset": "WinoGrande", "short": "WinoGrande", "metric": "accuracy", "explainer": "Fills a blank with one of two options using common sense, e.g. who \"got the easier cases\".", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 1267, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/allenai/winogrande"}], "gold": 1.2, "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1267 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · WinoGrande · median_ms | 118.90 ms | Setup & sourceHF text 0.4 · WinoGrande · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · WinoGrande · Raw index component | 93.53 | Setup & sourceHF text 0.4 · WinoGrande · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9353, "skill": 0.8706, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6883, "private_eq": 0.8454} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1267 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · WinoGrande · Chance-corrected public skill | 87.06 | Setup & sourceHF text 0.4 · WinoGrande · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9353, "skill": 0.8706, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6883, "private_eq": 0.8454} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1267 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · WinoGrande · Combined public/private component | 68.83 | Setup & sourceHF text 0.4 · WinoGrande · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9353, "skill": 0.8706, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6883, "private_eq": 0.8454} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · WinoGrande · Equated private same-skill component | 84.54 | Setup & sourceHF text 0.4 · WinoGrande · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9353, "skill": 0.8706, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6883, "private_eq": 0.8454} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HellaSwag · accuracy | 97.25% | Setup & sourceHF text 0.4 · HellaSwag · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9725, "jev": null, "metric": "accuracy", "requests": 10042, "answered": 10042, "unsupported": 0, "errors": 0, "median_ms": 120.3}. Benchmark contract: {"id": 29, "dataset": "HellaSwag", "short": "HellaSwag", "metric": "accuracy", "explainer": "Picks the most sensible of four continuations of an everyday scene.", "baseline": {"value": 0.25, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 10042, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/Rowan/hellaswag"}], "gold": 1.2, "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 10042 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HellaSwag · median_ms | 120.30 ms | Setup & sourceHF text 0.4 · HellaSwag · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HellaSwag · Raw index component | 97.25 | Setup & sourceHF text 0.4 · HellaSwag · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9725, "skill": 0.9633, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 10042 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HellaSwag · Chance-corrected public skill | 96.33 | Setup & sourceHF text 0.4 · HellaSwag · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9725, "skill": 0.9633, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 10042 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HellaSwag · Combined public/private component | 100.10 | Setup & sourceHF text 0.4 · HellaSwag · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9725, "skill": 0.9633, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HellaSwag · Equated private same-skill component | 100.00 | Setup & sourceHF text 0.4 · HellaSwag · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.9725, "skill": 0.9633, "coverage": 1, "random": 0.25, "tracks": [], "combined": 1.001, "private_eq": 1} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GSM8K · accuracy (4- and 10-choice) | 75.02% | Setup & sourceHF text 0.4 · GSM8K · accuracy (4- and 10-choice) Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7502, "jev": 0.7301, "metric": "accuracy (4- and 10-choice)", "requests": 2638, "answered": 2638, "unsupported": 0, "errors": 0, "median_ms": null}. Benchmark contract: {"id": 30, "dataset": "GSM8K", "short": "GSM8K", "metric": "accuracy", "explainer": "Picks a math word problem's answer from 4, and separately 10, numbers, with no working. Rebuilt for v0.3.", "baseline": {"value": 0.175, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 2638, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/openai/gsm8k"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2638 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GSM8K · Raw index component | 75.02 | Setup & sourceHF text 0.4 · GSM8K · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7502, "skill": 0.7021, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.8173, "random": 0.25}, {"track": "10-choice", "raw": 0.6831, "random": 0.1}], "combined": 0.7228, "private_eq": 0.7731} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2638 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GSM8K · Chance-corrected public skill | 70.21 | Setup & sourceHF text 0.4 · GSM8K · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7502, "skill": 0.7021, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.8173, "random": 0.25}, {"track": "10-choice", "raw": 0.6831, "random": 0.1}], "combined": 0.7228, "private_eq": 0.7731} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2638 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GSM8K · Combined public/private component | 72.28 | Setup & sourceHF text 0.4 · GSM8K · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7502, "skill": 0.7021, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.8173, "random": 0.25}, {"track": "10-choice", "raw": 0.6831, "random": 0.1}], "combined": 0.7228, "private_eq": 0.7731} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GSM8K · Equated private same-skill component | 77.31 | Setup & sourceHF text 0.4 · GSM8K · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7502, "skill": 0.7021, "coverage": 1, "random": 0.175, "tracks": [{"track": "4-choice", "raw": 0.8173, "random": 0.25}, {"track": "10-choice", "raw": 0.6831, "random": 0.1}], "combined": 0.7228, "private_eq": 0.7731} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GSM8K · 4-choice | 81.73% | Setup & sourceHF text 0.4 · GSM8K · 4-choice Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "4-choice", "raw": 0.8173, "random": 0.25} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · GSM8K · 10-choice | 68.31% | Setup & sourceHF text 0.4 · GSM8K · 10-choice Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "10-choice", "raw": 0.6831, "random": 0.1} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ChessBench · accuracy | 18.96% | Setup & sourceHF text 0.4 · ChessBench · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.1896, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 311.2}. Benchmark contract: {"id": 31, "dataset": "ChessBench", "short": "ChessBench", "metric": "accuracy", "explainer": "Given a chess position and every legal move, picks the strongest one; tied best moves count.", "baseline": {"value": 0.0819, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 5000, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2402.04494"}, {"kind": "code", "url": "https://github.com/google-deepmind/searchless_chess"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ChessBench · median_ms | 311.20 ms | Setup & sourceHF text 0.4 · ChessBench · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ChessBench · Raw index component | 18.96 | Setup & sourceHF text 0.4 · ChessBench · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.1896, "skill": 0.1174, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.0529, "private_eq": 0.1395} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ChessBench · Chance-corrected public skill | 11.74 | Setup & sourceHF text 0.4 · ChessBench · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.1896, "skill": 0.1174, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.0529, "private_eq": 0.1395} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ChessBench · Combined public/private component | 5.29 | Setup & sourceHF text 0.4 · ChessBench · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.1896, "skill": 0.1174, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.0529, "private_eq": 0.1395} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ChessBench · Equated private same-skill component | 13.95 | Setup & sourceHF text 0.4 · ChessBench · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.1896, "skill": 0.1174, "coverage": 1, "random": 0.0819, "tracks": [], "combined": 0.0529, "private_eq": 0.1395} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MuSR · accuracy | 66.76% | Setup & sourceHF text 0.4 · MuSR · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6676, "jev": null, "metric": "accuracy", "requests": 752, "answered": 752, "unsupported": 0, "errors": 0, "median_ms": 260.3}. Benchmark contract: {"id": 32, "dataset": "MuSR", "short": "MuSR", "metric": "accuracy", "explainer": "Reads a long story and answers one question needing clues from across it, e.g. who the murderer is.", "baseline": {"value": 0.371, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 752, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2310.16049"}, {"kind": "dataset", "url": "https://github.com/Zayne-sprague/MuSR"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 752 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MuSR · median_ms | 260.30 ms | Setup & sourceHF text 0.4 · MuSR · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MuSR · Raw index component | 66.76 | Setup & sourceHF text 0.4 · MuSR · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6676, "skill": 0.4715, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.3589, "private_eq": 0.6007} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 752 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MuSR · Chance-corrected public skill | 47.15 | Setup & sourceHF text 0.4 · MuSR · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6676, "skill": 0.4715, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.3589, "private_eq": 0.6007} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 752 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MuSR · Combined public/private component | 35.89 | Setup & sourceHF text 0.4 · MuSR · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6676, "skill": 0.4715, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.3589, "private_eq": 0.6007} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MuSR · Equated private same-skill component | 60.07 | Setup & sourceHF text 0.4 · MuSR · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6676, "skill": 0.4715, "coverage": 1, "random": 0.371, "tracks": [], "combined": 0.3589, "private_eq": 0.6007} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · SATA-Bench · case exact accuracy | 29.82% | Setup & sourceHF text 0.4 · SATA-Bench · case exact accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.2982, "jev": null, "metric": "case exact accuracy", "requests": 1650, "answered": 1650, "unsupported": 0, "errors": 0, "median_ms": 236.1}. Benchmark contract: {"id": 33, "dataset": "SATA-Bench", "short": "SATA-Bench", "metric": "case exact accuracy", "explainer": "Given a passage, a question and candidate answers, says yes or no to each; all must be right.", "baseline": {"value": 0.0131, "label": "Random whole-case guesses"}, "lower": false, "range": null, "cases": 1650, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/sata-bench/sata-bench"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1650 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · SATA-Bench · median_ms | 236.10 ms | Setup & sourceHF text 0.4 · SATA-Bench · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · SATA-Bench · Raw index component | 29.82 | Setup & sourceHF text 0.4 · SATA-Bench · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.2982, "skill": 0.2889, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.3022, "private_eq": 0.3182} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1650 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · SATA-Bench · Chance-corrected public skill | 28.89 | Setup & sourceHF text 0.4 · SATA-Bench · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.2982, "skill": 0.2889, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.3022, "private_eq": 0.3182} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1650 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · SATA-Bench · Combined public/private component | 30.22 | Setup & sourceHF text 0.4 · SATA-Bench · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.2982, "skill": 0.2889, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.3022, "private_eq": 0.3182} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · SATA-Bench · Equated private same-skill component | 31.82 | Setup & sourceHF text 0.4 · SATA-Bench · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native case exact accuracy. Source normalization: {"raw": 0.2982, "skill": 0.2889, "coverage": 1, "random": 0.0131, "tracks": [], "combined": 0.3022, "private_eq": 0.3182} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BRIGHT · nDCG@10 | 50.11% | Setup & sourceHF text 0.4 · BRIGHT · nDCG@10 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.5011, "jev": null, "metric": "nDCG@10", "requests": 220, "answered": 220, "unsupported": 0, "errors": 0, "median_ms": 3762.9}. Benchmark contract: {"id": 36, "dataset": "BRIGHT", "short": "BRIGHT", "metric": "nDCG@10", "explainer": "Rates how helpful each of 32 search-hit documents is for a hard question; ranked by nDCG@10, on questions where at least one of the 32 is relevant.", "baseline": {"value": 0.116, "label": "Random ranking, answerable queries", "previous": 0.0448}, "lower": false, "range": null, "cases": 220, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2407.12883"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/xlangai/BRIGHT"}], "gold": 1.2, "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 220 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BRIGHT · median_ms | 3762.90 ms | Setup & sourceHF text 0.4 · BRIGHT · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BRIGHT · Raw index component | 50.11 | Setup & sourceHF text 0.4 · BRIGHT · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.5011, "skill": 0.4356, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4087, "private_eq": 0.4824} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 220 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BRIGHT · Chance-corrected public skill | 43.56 | Setup & sourceHF text 0.4 · BRIGHT · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.5011, "skill": 0.4356, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4087, "private_eq": 0.4824} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 220 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BRIGHT · Combined public/private component | 40.87 | Setup & sourceHF text 0.4 · BRIGHT · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.5011, "skill": 0.4356, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4087, "private_eq": 0.4824} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BRIGHT · Equated private same-skill component | 48.24 | Setup & sourceHF text 0.4 · BRIGHT · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native nDCG@10. Source normalization: {"raw": 0.5011, "skill": 0.4356, "coverage": 1, "random": 0.116, "tracks": [], "combined": 0.4087, "private_eq": 0.4824} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Amazon ESCI · macro-F1 | 62.09% | Setup & sourceHF text 0.4 · Amazon ESCI · macro-F1 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6209, "jev": null, "metric": "macro-F1", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 123.1}. Benchmark contract: {"id": 37, "dataset": "Amazon ESCI", "short": "Amazon ESCI", "metric": "macro-F1", "explainer": "Labels a product as an exact match, substitute, complement or irrelevant to a shopping query.", "baseline": {"value": 0.2027, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 5000, "group": "retrieval", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/amazon-science/esci-data"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Amazon ESCI · median_ms | 123.10 ms | Setup & sourceHF text 0.4 · Amazon ESCI · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Amazon ESCI · Raw index component | 62.09 | Setup & sourceHF text 0.4 · Amazon ESCI · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.6209, "skill": 0.5245, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.6219, "private_eq": 0.7012} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Amazon ESCI · Chance-corrected public skill | 52.45 | Setup & sourceHF text 0.4 · Amazon ESCI · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.6209, "skill": 0.5245, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.6219, "private_eq": 0.7012} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Amazon ESCI · Combined public/private component | 62.19 | Setup & sourceHF text 0.4 · Amazon ESCI · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.6209, "skill": 0.5245, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.6219, "private_eq": 0.7012} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Amazon ESCI · Equated private same-skill component | 70.12 | Setup & sourceHF text 0.4 · Amazon ESCI · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.6209, "skill": 0.5245, "coverage": 1, "random": 0.2027, "tracks": [], "combined": 0.6219, "private_eq": 0.7012} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ACOS · per-review F1 | 45.12% | Setup & sourceHF text 0.4 · ACOS · per-review F1 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.4512, "jev": null, "metric": "per-review F1", "requests": 1565, "answered": 1565, "unsupported": 0, "errors": 0, "median_ms": 1808.6}. Benchmark contract: {"id": 38, "dataset": "ACOS", "short": "ACOS", "metric": "per-review F1", "explainer": "Reads a restaurant or laptop review and says yes/no for each aspect-and-sentiment pair; scored by F1 per review, averaged over reviews.", "baseline": {"value": 0.031, "label": "Always yes · per-review F1", "previous": 0}, "lower": false, "range": null, "cases": 400, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2021.acl-long.29"}, {"kind": "dataset", "url": "https://github.com/NUSTM/ACOS"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1565 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ACOS · median_ms | 1808.60 ms | Setup & sourceHF text 0.4 · ACOS · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ACOS · Raw index component | 45.12 | Setup & sourceHF text 0.4 · ACOS · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.4512, "skill": 0.4336, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.6277, "private_eq": 0.6424} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1565 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ACOS · Chance-corrected public skill | 43.36 | Setup & sourceHF text 0.4 · ACOS · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.4512, "skill": 0.4336, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.6277, "private_eq": 0.6424} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1565 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ACOS · Combined public/private component | 62.77 | Setup & sourceHF text 0.4 · ACOS · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.4512, "skill": 0.4336, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.6277, "private_eq": 0.6424} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · ACOS · Equated private same-skill component | 64.24 | Setup & sourceHF text 0.4 · ACOS · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native per-review F1. Source normalization: {"raw": 0.4512, "skill": 0.4336, "coverage": 1, "random": 0.031, "tracks": [], "combined": 0.6277, "private_eq": 0.6424} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · FinEntity · macro-F1 | 94.22% | Setup & sourceHF text 0.4 · FinEntity · macro-F1 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.9422, "jev": null, "metric": "macro-F1", "requests": 979, "answered": 979, "unsupported": 0, "errors": 0, "median_ms": 120.5}. Benchmark contract: {"id": 39, "dataset": "FinEntity", "short": "FinEntity", "metric": "macro-F1", "explainer": "Says whether a financial news snippet is positive, negative or neutral about a named firm.", "baseline": {"value": 0.3201, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 979, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2023.emnlp-main.956"}, {"kind": "dataset", "url": "https://github.com/yixuantt/FinEntity"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 979 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · FinEntity · median_ms | 120.50 ms | Setup & sourceHF text 0.4 · FinEntity · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · FinEntity · Raw index component | 94.22 | Setup & sourceHF text 0.4 · FinEntity · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9422, "skill": 0.915, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5736, "private_eq": 0.7127} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 979 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · FinEntity · Chance-corrected public skill | 91.50 | Setup & sourceHF text 0.4 · FinEntity · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9422, "skill": 0.915, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5736, "private_eq": 0.7127} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 979 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · FinEntity · Combined public/private component | 57.36 | Setup & sourceHF text 0.4 · FinEntity · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9422, "skill": 0.915, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5736, "private_eq": 0.7127} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · FinEntity · Equated private same-skill component | 71.27 | Setup & sourceHF text 0.4 · FinEntity · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.9422, "skill": 0.915, "coverage": 1, "random": 0.3201, "tracks": [], "combined": 0.5736, "private_eq": 0.7127} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · iSarcasmEval · Sarcasm F1 · track A, English | 73.30% | Setup & sourceHF text 0.4 · iSarcasmEval · Sarcasm F1 · track A, English Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.733, "jev": null, "metric": "Sarcasm F1 \u00b7 track A, English", "requests": 4600, "answered": 4600, "unsupported": 0, "errors": 0, "median_ms": 119.2}. Benchmark contract: {"id": 40, "dataset": "iSarcasmEval", "short": "iSarcasmEval", "metric": "Sarcasm F1 · track A, English", "explainer": "Says yes or no: is this tweet sarcastic? Scored with F1 on sarcastic English tweets (track A); other tracks listed under each result.", "baseline": {"value": 0.2227, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 4600, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://aclanthology.org/2022.semeval-1.111"}, {"kind": "dataset", "url": "https://github.com/iabufarha/iSarcasmEval"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 4600 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · iSarcasmEval · median_ms | 119.20 ms | Setup & sourceHF text 0.4 · iSarcasmEval · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · iSarcasmEval · Raw index component | 73.30 | Setup & sourceHF text 0.4 · iSarcasmEval · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.733, "skill": 0.6565, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.589, "headline": false}, {"track": "A · English", "score": 0.733, "headline": true}, {"track": "C · Arabic pairs", "score": 0.955, "headline": false}, {"track": "C · English pairs", "score": 0.98, "headline": false}], "combined": 0.6637, "private_eq": 0.7408} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 4600 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · iSarcasmEval · Chance-corrected public skill | 65.65 | Setup & sourceHF text 0.4 · iSarcasmEval · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.733, "skill": 0.6565, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.589, "headline": false}, {"track": "A · English", "score": 0.733, "headline": true}, {"track": "C · Arabic pairs", "score": 0.955, "headline": false}, {"track": "C · English pairs", "score": 0.98, "headline": false}], "combined": 0.6637, "private_eq": 0.7408} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 4600 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · iSarcasmEval · Combined public/private component | 66.37 | Setup & sourceHF text 0.4 · iSarcasmEval · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.733, "skill": 0.6565, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.589, "headline": false}, {"track": "A · English", "score": 0.733, "headline": true}, {"track": "C · Arabic pairs", "score": 0.955, "headline": false}, {"track": "C · English pairs", "score": 0.98, "headline": false}], "combined": 0.6637, "private_eq": 0.7408} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · iSarcasmEval · Equated private same-skill component | 74.08 | Setup & sourceHF text 0.4 · iSarcasmEval · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native Sarcasm F1 · track A, English. Source normalization: {"raw": 0.733, "skill": 0.6565, "coverage": 1, "random": 0.2227, "tracks": [{"track": "A · Arabic", "score": 0.589, "headline": false}, {"track": "A · English", "score": 0.733, "headline": true}, {"track": "C · Arabic pairs", "score": 0.955, "headline": false}, {"track": "C · English pairs", "score": 0.98, "headline": false}], "combined": 0.6637, "private_eq": 0.7408} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · iSarcasmEval · A · Arabic | 58.90% | Setup & sourceHF text 0.4 · iSarcasmEval · A · Arabic Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "A · Arabic", "score": 0.589, "headline": false} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · iSarcasmEval · C · Arabic pairs | 95.50% | Setup & sourceHF text 0.4 · iSarcasmEval · C · Arabic pairs Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "C · Arabic pairs", "score": 0.955, "headline": false} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · iSarcasmEval · C · English pairs | 98.00% | Setup & sourceHF text 0.4 · iSarcasmEval · C · English pairs Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published separate track; sample count unspecified. {"track": "C · English pairs", "score": 0.98, "headline": false} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · VAST · macro-F1 | 78.17% | Setup & sourceHF text 0.4 · VAST · macro-F1 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7817, "jev": null, "metric": "macro-F1", "requests": 3006, "answered": 3006, "unsupported": 0, "errors": 0, "median_ms": 119.3}. Benchmark contract: {"id": 41, "dataset": "VAST", "short": "VAST", "metric": "macro-F1", "explainer": "Says whether a comment is for, against, or neutral on a given topic.", "baseline": {"value": 0.3333, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 3006, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/emilyallaway/zero-shot-stance"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3006 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · VAST · median_ms | 119.30 ms | Setup & sourceHF text 0.4 · VAST · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · VAST · Raw index component | 78.17 | Setup & sourceHF text 0.4 · VAST · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7817, "skill": 0.6726, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.4788, "private_eq": 0.6558} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3006 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · VAST · Chance-corrected public skill | 67.26 | Setup & sourceHF text 0.4 · VAST · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7817, "skill": 0.6726, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.4788, "private_eq": 0.6558} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3006 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · VAST · Combined public/private component | 47.88 | Setup & sourceHF text 0.4 · VAST · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7817, "skill": 0.6726, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.4788, "private_eq": 0.6558} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · VAST · Equated private same-skill component | 65.58 | Setup & sourceHF text 0.4 · VAST · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.7817, "skill": 0.6726, "coverage": 1, "random": 0.3333, "tracks": [], "combined": 0.4788, "private_eq": 0.6558} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · NLI4CT · macro-F1 | 86.82% | Setup & sourceHF text 0.4 · NLI4CT · macro-F1 Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8682, "jev": null, "metric": "macro-F1", "requests": 5500, "answered": 5500, "unsupported": 0, "errors": 0, "median_ms": 156}. Benchmark contract: {"id": 42, "dataset": "NLI4CT", "short": "NLI4CT", "metric": "macro-F1", "explainer": "Says whether clinical-trial report sections support or contradict a statement, some deliberately altered.", "baseline": {"value": 0.486, "label": "Random choices · F1"}, "lower": false, "range": null, "cases": 5500, "group": "language", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/ai-systems/Task-2-SemEval-2024"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5500 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · NLI4CT · median_ms | 156.00 ms | Setup & sourceHF text 0.4 · NLI4CT · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · NLI4CT · Raw index component | 86.82 | Setup & sourceHF text 0.4 · NLI4CT · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8682, "skill": 0.7436, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.9335, "private_eq": 0.9657} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5500 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · NLI4CT · Chance-corrected public skill | 74.36 | Setup & sourceHF text 0.4 · NLI4CT · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8682, "skill": 0.7436, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.9335, "private_eq": 0.9657} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5500 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · NLI4CT · Combined public/private component | 93.35 | Setup & sourceHF text 0.4 · NLI4CT · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8682, "skill": 0.7436, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.9335, "private_eq": 0.9657} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · NLI4CT · Equated private same-skill component | 96.57 | Setup & sourceHF text 0.4 · NLI4CT · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native macro-F1. Source normalization: {"raw": 0.8682, "skill": 0.7436, "coverage": 1, "random": 0.486, "tracks": [], "combined": 0.9335, "private_eq": 0.9657} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CRUXEval · accuracy | 86.84% | Setup & sourceHF text 0.4 · CRUXEval · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8684, "jev": null, "metric": "accuracy", "requests": 570, "answered": 570, "unsupported": 0, "errors": 0, "median_ms": 118.2}. Benchmark contract: {"id": 43, "dataset": "CRUXEval", "short": "CRUXEval", "metric": "accuracy", "explainer": "Reads a short Python function and an input and picks its output from 2 to 10 candidates, without running it.", "baseline": {"value": 0.3697, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 570, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2401.03065"}, {"kind": "dataset", "url": "https://github.com/facebookresearch/cruxeval"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 570 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CRUXEval · median_ms | 118.20 ms | Setup & sourceHF text 0.4 · CRUXEval · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CRUXEval · Raw index component | 86.84 | Setup & sourceHF text 0.4 · CRUXEval · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8684, "skill": 0.7912, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.6165, "private_eq": 0.7604} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 570 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CRUXEval · Chance-corrected public skill | 79.12 | Setup & sourceHF text 0.4 · CRUXEval · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8684, "skill": 0.7912, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.6165, "private_eq": 0.7604} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 570 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CRUXEval · Combined public/private component | 61.65 | Setup & sourceHF text 0.4 · CRUXEval · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8684, "skill": 0.7912, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.6165, "private_eq": 0.7604} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CRUXEval · Equated private same-skill component | 76.04 | Setup & sourceHF text 0.4 · CRUXEval · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8684, "skill": 0.7912, "coverage": 1, "random": 0.3697, "tracks": [], "combined": 0.6165, "private_eq": 0.7604} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLadder · accuracy | 80.54% | Setup & sourceHF text 0.4 · CLadder · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8054, "jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 118.3}. Benchmark contract: {"id": 44, "dataset": "CLadder", "short": "CLadder", "metric": "accuracy", "explainer": "Answers a yes/no question about cause and effect in a small made-up world with stated probabilities.", "baseline": {"value": 0.5, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 5000, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2312.04350"}, {"kind": "dataset", "url": "https://github.com/causalNLP/cladder"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLadder · median_ms | 118.30 ms | Setup & sourceHF text 0.4 · CLadder · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLadder · Raw index component | 80.54 | Setup & sourceHF text 0.4 · CLadder · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8054, "skill": 0.6108, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.5511, "private_eq": 0.7776} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLadder · Chance-corrected public skill | 61.08 | Setup & sourceHF text 0.4 · CLadder · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8054, "skill": 0.6108, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.5511, "private_eq": 0.7776} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLadder · Combined public/private component | 55.11 | Setup & sourceHF text 0.4 · CLadder · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8054, "skill": 0.6108, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.5511, "private_eq": 0.7776} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · CLadder · Equated private same-skill component | 77.76 | Setup & sourceHF text 0.4 · CLadder · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8054, "skill": 0.6108, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.5511, "private_eq": 0.7776} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HLE · accuracy | 12.77% | Setup & sourceHF text 0.4 · HLE · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.1277, "jev": null, "metric": "accuracy", "requests": 501, "answered": 501, "unsupported": 0, "errors": 0, "median_ms": 120.5}. Benchmark contract: {"id": 45, "dataset": "HLE", "short": "HLE", "metric": "accuracy", "explainer": "Expert-level multiple choice from Humanity's Last Exam, text-only questions, 5 to 26 options.", "baseline": {"value": 0.1641, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 501, "group": "knowledge", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/cais/hle"}], "gold": 1.2} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 501 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HLE · median_ms | 120.50 ms | Setup & sourceHF text 0.4 · HLE · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HLE · Raw index component | 12.77 | Setup & sourceHF text 0.4 · HLE · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.1277, "skill": 0, "coverage": 1, "random": 0.1641, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 501 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HLE · Chance-corrected public skill | 0.00 | Setup & sourceHF text 0.4 · HLE · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.1277, "skill": 0, "coverage": 1, "random": 0.1641, "tracks": []} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 501 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Habermas Machine · accuracy | 40.04% | Setup & sourceHF text 0.4 · Habermas Machine · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.4004, "jev": null, "metric": "accuracy", "requests": 1676, "answered": 1676, "unsupported": 0, "errors": 0, "median_ms": 236.5}. Benchmark contract: {"id": 50, "dataset": "Habermas Machine", "short": "Habermas", "metric": "accuracy", "explainer": "Given a group's opinions on a policy question, predicts which candidate statement the group ranked highest.", "baseline": {"value": 0.311, "label": "Uniform random choice"}, "lower": false, "range": null, "cases": 1676, "group": "arts", "in_suite": true, "interactive": false, "references": [{"kind": "dataset", "url": "https://github.com/google-deepmind/habermas_machine"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1676 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Habermas Machine · median_ms | 236.50 ms | Setup & sourceHF text 0.4 · Habermas Machine · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Habermas Machine · Raw index component | 40.04 | Setup & sourceHF text 0.4 · Habermas Machine · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.4004, "skill": 0.1297, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.2031, "private_eq": 0.4565} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1676 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Habermas Machine · Chance-corrected public skill | 12.97 | Setup & sourceHF text 0.4 · Habermas Machine · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.4004, "skill": 0.1297, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.2031, "private_eq": 0.4565} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 1676 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Habermas Machine · Combined public/private component | 20.31 | Setup & sourceHF text 0.4 · Habermas Machine · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.4004, "skill": 0.1297, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.2031, "private_eq": 0.4565} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · Habermas Machine · Equated private same-skill component | 45.65 | Setup & sourceHF text 0.4 · Habermas Machine · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.4004, "skill": 0.1297, "coverage": 1, "random": 0.311, "tracks": [], "combined": 0.2031, "private_eq": 0.4565} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · PhishNChips phishing decisions · accuracy | 78.40% | Setup & sourceHF text 0.4 · PhishNChips phishing decisions · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.784, "jev": null, "metric": "accuracy", "requests": 2000, "answered": 2000, "unsupported": 0, "errors": 0, "median_ms": 181.8}. Benchmark contract: {"id": 56, "dataset": "PhishNChips phishing decisions", "short": "PhishNChips", "metric": "accuracy", "explainer": "Says whether an email and its link are phishing or legitimate.", "baseline": {"value": 0.5, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 2000, "group": "retrieval", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "dataset", "url": "https://huggingface.co/datasets/AreLit/PhishNChips"}, {"kind": "code", "url": "https://github.com/anisselbd/jev-phishing-bench"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · PhishNChips phishing decisions · median_ms | 181.80 ms | Setup & sourceHF text 0.4 · PhishNChips phishing decisions · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · PhishNChips phishing decisions · Raw index component | 78.40 | Setup & sourceHF text 0.4 · PhishNChips phishing decisions · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.784, "skill": 0.568, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7241, "private_eq": 0.8631} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · PhishNChips phishing decisions · Chance-corrected public skill | 56.80 | Setup & sourceHF text 0.4 · PhishNChips phishing decisions · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.784, "skill": 0.568, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7241, "private_eq": 0.8631} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · PhishNChips phishing decisions · Combined public/private component | 72.41 | Setup & sourceHF text 0.4 · PhishNChips phishing decisions · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.784, "skill": 0.568, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7241, "private_eq": 0.8631} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · PhishNChips phishing decisions · Equated private same-skill component | 86.31 | Setup & sourceHF text 0.4 · PhishNChips phishing decisions · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.784, "skill": 0.568, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.7241, "private_eq": 0.8631} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MMLU-Pro · accuracy | 68.04% | Setup & sourceHF text 0.4 · MMLU-Pro · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.6804, "jev": null, "metric": "accuracy", "requests": 12032, "answered": 12032, "unsupported": 0, "errors": 0, "median_ms": 120.6}. Benchmark contract: {"id": 57, "dataset": "MMLU-Pro", "short": "MMLU-Pro", "metric": "accuracy", "explainer": "Harder MMLU successor: questions across 14 subjects with up to ten options.", "baseline": {"value": 0.1109, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 12032, "group": "knowledge", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2406.01574"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro"}], "gold": 1.2, "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 12032 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MMLU-Pro · median_ms | 120.60 ms | Setup & sourceHF text 0.4 · MMLU-Pro · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MMLU-Pro · Raw index component | 68.04 | Setup & sourceHF text 0.4 · MMLU-Pro · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6804, "skill": 0.6405, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6221, "combined": 0.5711} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 12032 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MMLU-Pro · Chance-corrected public skill | 64.05 | Setup & sourceHF text 0.4 · MMLU-Pro · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6804, "skill": 0.6405, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6221, "combined": 0.5711} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 12032 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MMLU-Pro · Combined public/private component | 57.11 | Setup & sourceHF text 0.4 · MMLU-Pro · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6804, "skill": 0.6405, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6221, "combined": 0.5711} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · MMLU-Pro · Equated private same-skill component | 62.21 | Setup & sourceHF text 0.4 · MMLU-Pro · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.6804, "skill": 0.6405, "coverage": 1, "random": 0.1109, "tracks": [], "private_eq": 0.6221, "combined": 0.5711} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BBH · accuracy | 87.91% | Setup & sourceHF text 0.4 · BBH · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8791, "jev": null, "metric": "accuracy", "requests": 5507, "answered": 5507, "unsupported": 0, "errors": 0, "median_ms": 120.6}. Benchmark contract: {"id": 58, "dataset": "BBH", "short": "BBH", "metric": "accuracy", "explainer": "23 tricky reasoning tasks from BIG-Bench (logic, dates, tracking objects, fallacies), answered with no working.", "baseline": {"value": 0.3101, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 5507, "group": "knowledge", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2210.09261"}, {"kind": "dataset", "url": "https://github.com/suzgunmirac/BIG-Bench-Hard"}], "gold": 1.2, "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5507 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BBH · median_ms | 120.60 ms | Setup & sourceHF text 0.4 · BBH · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BBH · Raw index component | 87.91 | Setup & sourceHF text 0.4 · BBH · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8791, "skill": 0.8248, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.5199, "combined": 0.2971} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5507 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BBH · Chance-corrected public skill | 82.48 | Setup & sourceHF text 0.4 · BBH · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8791, "skill": 0.8248, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.5199, "combined": 0.2971} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 5507 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BBH · Combined public/private component | 29.71 | Setup & sourceHF text 0.4 · BBH · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8791, "skill": 0.8248, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.5199, "combined": 0.2971} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · BBH · Equated private same-skill component | 51.99 | Setup & sourceHF text 0.4 · BBH · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.8791, "skill": 0.8248, "coverage": 1, "random": 0.3101, "tracks": [], "private_eq": 0.5199, "combined": 0.2971} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · RAGTruth · F1 on hallucinated class | 83.26% | Setup & sourceHF text 0.4 · RAGTruth · F1 on hallucinated class Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.8326, "jev": null, "metric": "F1 on hallucinated class", "requests": 2700, "answered": 2700, "unsupported": 0, "errors": 0, "median_ms": 124.1}. Benchmark contract: {"id": 59, "dataset": "RAGTruth", "short": "RAGTruth", "metric": "F1 on hallucinated class", "explainer": "Says yes or no: does an AI-written answer contain anything its source text doesn't support? Scored with F1.", "baseline": {"value": 0.5177, "label": "Always \"hallucinated\" · F1"}, "lower": false, "range": null, "cases": 2700, "group": "language", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "dataset", "url": "https://github.com/ParticleMedia/RAGTruth"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2700 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · RAGTruth · median_ms | 124.10 ms | Setup & sourceHF text 0.4 · RAGTruth · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · RAGTruth · Raw index component | 83.26 | Setup & sourceHF text 0.4 · RAGTruth · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.8326, "skill": 0.6529, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.3542, "private_eq": 0.6916} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2700 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · RAGTruth · Chance-corrected public skill | 65.29 | Setup & sourceHF text 0.4 · RAGTruth · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.8326, "skill": 0.6529, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.3542, "private_eq": 0.6916} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 2700 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · RAGTruth · Combined public/private component | 35.42 | Setup & sourceHF text 0.4 · RAGTruth · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.8326, "skill": 0.6529, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.3542, "private_eq": 0.6916} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · RAGTruth · Equated private same-skill component | 69.16 | Setup & sourceHF text 0.4 · RAGTruth · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native F1 on hallucinated class. Source normalization: {"raw": 0.8326, "skill": 0.6529, "coverage": 1, "random": 0.5177, "tracks": [], "combined": 0.3542, "private_eq": 0.6916} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HoVer · accuracy | 77.40% | Setup & sourceHF text 0.4 · HoVer · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.774, "jev": null, "metric": "accuracy", "requests": 4000, "answered": 4000, "unsupported": 0, "errors": 0, "median_ms": 135.3}. Benchmark contract: {"id": 61, "dataset": "HoVer", "short": "HoVer", "metric": "accuracy", "explainer": "Decides whether a claim linking two to four Wikipedia articles is supported by their introductions.", "baseline": {"value": 0.5, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 4000, "group": "retrieval", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2011.03088"}, {"kind": "dataset", "url": "https://github.com/hover-nlp/hover"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 4000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HoVer · median_ms | 135.30 ms | Setup & sourceHF text 0.4 · HoVer · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HoVer · Raw index component | 77.40 | Setup & sourceHF text 0.4 · HoVer · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.774, "skill": 0.548, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6266, "private_eq": 0.8149} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 4000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HoVer · Chance-corrected public skill | 54.80 | Setup & sourceHF text 0.4 · HoVer · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.774, "skill": 0.548, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6266, "private_eq": 0.8149} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 4000 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HoVer · Combined public/private component | 62.66 | Setup & sourceHF text 0.4 · HoVer · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.774, "skill": 0.548, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6266, "private_eq": 0.8149} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · HoVer · Equated private same-skill component | 81.49 | Setup & sourceHF text 0.4 · HoVer · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.774, "skill": 0.548, "coverage": 1, "random": 0.5, "tracks": [], "combined": 0.6266, "private_eq": 0.8149} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · When2Call · accuracy | 71.77% | Setup & sourceHF text 0.4 · When2Call · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7177, "jev": null, "metric": "accuracy", "requests": 3652, "answered": 3652, "unsupported": 0, "errors": 0, "median_ms": 264.3}. Benchmark contract: {"id": 62, "dataset": "When2Call", "short": "When2Call", "metric": "accuracy", "explainer": "Given a question and an assistant's tools, picks the right move: answer, call a tool, ask for details, or decline.", "baseline": {"value": 0.25, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 3652, "group": "tools", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://aclanthology.org/2025.naacl-long.174"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/nvidia/When2Call"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3652 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · When2Call · median_ms | 264.30 ms | Setup & sourceHF text 0.4 · When2Call · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · When2Call · Raw index component | 71.77 | Setup & sourceHF text 0.4 · When2Call · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7177, "skill": 0.6236, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.5767, "private_eq": 0.6854} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3652 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · When2Call · Chance-corrected public skill | 62.36 | Setup & sourceHF text 0.4 · When2Call · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7177, "skill": 0.6236, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.5767, "private_eq": 0.6854} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 3652 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · When2Call · Combined public/private component | 57.67 | Setup & sourceHF text 0.4 · When2Call · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7177, "skill": 0.6236, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.5767, "private_eq": 0.6854} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · When2Call · Equated private same-skill component | 68.54 | Setup & sourceHF text 0.4 · When2Call · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7177, "skill": 0.6236, "coverage": 1, "random": 0.25, "tracks": [], "combined": 0.5767, "private_eq": 0.6854} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · New Yorker · accuracy | 75.38% | Setup & sourceHF text 0.4 · New Yorker · accuracy Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Native source result: {"score": 0.7538, "jev": null, "metric": "accuracy", "requests": 528, "answered": 528, "unsupported": 0, "errors": 0, "median_ms": 120.2}. Benchmark contract: {"id": 64, "dataset": "New Yorker", "short": "New Yorker", "metric": "accuracy", "explainer": "Given a text description of a New Yorker cartoon, picks the caption written for it from five.", "baseline": {"value": 0.2, "label": "uniform random choice, mean of 1/options"}, "lower": false, "range": null, "cases": 528, "group": "arts", "in_suite": true, "interactive": false, "edition_added": "v0.2", "references": [{"kind": "paper", "url": "https://arxiv.org/abs/2209.06293"}, {"kind": "dataset", "url": "https://huggingface.co/datasets/jmhessel/newyorker_caption_contest"}], "private": true} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 528 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · New Yorker · median_ms | 120.20 ms | Setup & sourceHF text 0.4 · New Yorker · median_ms Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Published measured timing; excludes null and zero batch placeholders. Median and p95 over the timed rows that returned an answer; per-benchmark medians are shown only where the sample holds at least 5 rows of that benchmark. A bootstrap on the full single-process runs of ten entrants put the sample median within 5% and the p95 within 10% of a full run at 95% confidence. Earlier board figures came from the scoring runs, some timed with 6 to 12 processes sharing a card or with queue waits removed; they are no longer shown. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · New Yorker · Raw index component | 75.38 | Setup & sourceHF text 0.4 · New Yorker · Raw index component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7538, "skill": 0.6923, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.7003, "private_eq": 0.7622} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 528 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · New Yorker · Chance-corrected public skill | 69.23 | Setup & sourceHF text 0.4 · New Yorker · Chance-corrected public skill Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7538, "skill": 0.6923, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.7003, "private_eq": 0.7622} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 528 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · New Yorker · Combined public/private component | 70.03 | Setup & sourceHF text 0.4 · New Yorker · Combined public/private component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7538, "skill": 0.6923, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.7003, "private_eq": 0.7622} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF text 0.4 · New Yorker · Equated private same-skill component | 76.22 | Setup & sourceHF text 0.4 · New Yorker · Equated private same-skill component Exact source engine amos-1-large; {"kind": "full fine-tune", "technique": "autoregressive", "technique_name": "autoregressive", "served_params": 27781427952, "params_estimated": false, "params_from_name": false, "trained_bytes": null, "model_url": "https://huggingface.co/mukaddamzaid/amos-1-large", "code_url": null, "base_model": "Qwen/Qwen3.8-27B", "base_url": "https://huggingface.co/Qwen/Qwen3.8-27B", "weights_repo": null, "served_checkpoint": null}. New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Decision Index 0.4.1 (release-v4), source generated 2026-11-06T09:05:08Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Source notes: New in the 2026-10-14 PR batch (PR #170). Submitted public 0.3 run adopted (kit rescore 64.94, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.80% on 831 rows (bar 98.5%). Served with --rotations 2 (answers averaged over two option orders), the author's submitted setting. Training data unpublished. Author public-run adoption/execution status and private lab setup are disclosed in configuration; evaluator-published does not mean every public run was independently executed. Normalized/index component, not native accuracy. Source normalization: {"raw": 0.7538, "skill": 0.6923, "coverage": 1, "random": 0.2, "tracks": [], "combined": 0.7003, "private_eq": 0.7622} The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Public | 76.10 | Setup & sourceHF vision 0.4 · Public Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Private | 68.71 | Setup & sourceHF vision 0.4 · Private Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Full | 72.40 | Setup & sourceHF vision 0.4 · Full Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · CV-Bench · pub | 77.33 | Setup & sourceHF vision 0.4 · CV-Bench · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · CV-Bench · private | 44.72 | Setup & sourceHF vision 0.4 · CV-Bench · private Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · BLINK · pub | 79.38 | Setup & sourceHF vision 0.4 · BLINK · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · BLINK · private | 58.67 | Setup & sourceHF vision 0.4 · BLINK · private Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · RealWorldQA · pub | 76.14 | Setup & sourceHF vision 0.4 · RealWorldQA · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · RealWorldQA · private | 44.00 | Setup & sourceHF vision 0.4 · RealWorldQA · private Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · CharXiv · pub | 79.95 | Setup & sourceHF vision 0.4 · CharXiv · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · CharXiv · private | 76.43 | Setup & sourceHF vision 0.4 · CharXiv · private Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · InfographicVQA · pub | 97.27 | Setup & sourceHF vision 0.4 · InfographicVQA · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · InfographicVQA · private | 99.33 | Setup & sourceHF vision 0.4 · InfographicVQA · private Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Mind2Web · pub | 82.19 | Setup & sourceHF vision 0.4 · Mind2Web · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Mind2Web · private | 66.67 | Setup & sourceHF vision 0.4 · Mind2Web · private Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Winoground · pub | 86.25 | Setup & sourceHF vision 0.4 · Winoground · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Winoground · private | 95.00 | Setup & sourceHF vision 0.4 · Winoground · private Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · KIE (CORD+FUNSD) · pub | 92.39 | Setup & sourceHF vision 0.4 · KIE (CORD+FUNSD) · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · KIE (CORD+FUNSD) · private | 86.60 | Setup & sourceHF vision 0.4 · KIE (CORD+FUNSD) · private Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Moderation (Hateful Memes) · pub | 42.79 | Setup & sourceHF vision 0.4 · Moderation (Hateful Memes) · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Moderation (Hateful Memes) · private | 47.00 | Setup & sourceHF vision 0.4 · Moderation (Hateful Memes) · private Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · R-Bench-M · pub | 33.76 | Setup & sourceHF vision 0.4 · R-Bench-M · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · MMMU-Pro vision · pub | 45.50 | Setup & sourceHF vision 0.4 · MMMU-Pro vision · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision calibration 0.4 · acc | 80.24% | Setup & sourceHF vision calibration 0.4 · acc Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Source calibration sample; no new binning assumptions. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 13686 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision calibration 0.4 · conf | 79.70% | Setup & sourceHF vision calibration 0.4 · conf Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Source calibration sample; no new binning assumptions. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 13686 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision calibration 0.4 · ece | 2.35 pp | Setup & sourceHF vision calibration 0.4 · ece Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Source calibration sample; no new binning assumptions. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 13686 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision calibration 0.4 · over95 | 0.32% | Setup & sourceHF vision calibration 0.4 · over95 Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Source calibration sample; no new binning assumptions. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: 13686 Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Row latency | 385.90 ms | Setup & sourceHF vision 0.4 · Row latency Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Native image budget · pub | 76.10 | Setup & sourceHF vision 0.4 · Native image budget · pub Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Heterogeneous native budget 1.6 MP; separate from fixed 1.6MP comparison. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Native image budget · priv | 68.71 | Setup & sourceHF vision 0.4 · Native image budget · priv Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Heterogeneous native budget 1.6 MP; separate from fixed 1.6MP comparison. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |
| HF vision 0.4 · Native image budget · full | 72.40 | Setup & sourceHF vision 0.4 · Native image budget · full Exact source engine amos-1-large; community vision 0.4. Board cap 1638400 pixels; effective 1.6 MP. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Community Decision Index vision 0.4; 50% public + 50% private, separate from text Full (60/40 private-only). Source generated 2026-11-06T09:05:12Z. Public/private benchmark values are source-normalized/chance-corrected index points and may be negative. No clamping or inferred raw accuracy. 26 requests over the author's 4 MiB body limit count as wrong (9 public, 17 private). Heterogeneous native budget 1.6 MP; separate from fixed 1.6MP comparison. The board entry links mukaddamzaid/amos-1-large and names no served checkpoint; the registry pins card d67e051, whose weights are unchanged since 13 October. Some vision requests over the author’s 4 MiB body limit count as wrong, per the board caveat. Decision Index 0.4 (release-v4), source generated 2026-10-20T14:13:56Z. Full = 60% equated private same-skill + 40% equated private new-domain tasks; public contributes zero. Both private components are mapped onto the public scale using untuned stock models. Public weighting/panel changed (GSM8K rebuilt, ForecastBench retired, WinoGrande area reassigned); these are new benchmark definitions, not proof of a model upgrade. Public 37-benchmark index and 42 published native benchmarks remain separate. Preserve source rounding, coverage, exclusions and unknown execution dates; no new uncertainty inferred. Checked / reported: 2026-11-08 · Samples: Not published Snapshot: 2026-10-20T14:13:56Z · Revision: 133313cf8963f2b6bed960abb0152fd14cbb6731; c130faf65f82e81d3bdefb01e758c712dc5c5199 for the 2 November 2026 additions; f0dbd5521d35156e694dc5ba43d553d43747681c for the 5 November 2026 additions; e05cd04cad2b4f8d6c34aaa486ff88f1a9584942 for the 7 November 2026 additions · Published observations; result reuse terms not specified Result source · Benchmark source |