Examenos

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 400

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 50

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 50

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 50

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 50

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 400

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 400

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 200

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 50

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 50

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 50

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

400 contrastive decisions in 200 pairs across eight workflow tasks; pair accuracy needs both answers correct.

{"model_requested": "drex-v2.0", "model_served": "drex-v2.0", "endpoint": "Nace.AI Drex /v1/systemone", "protocol": "few-shot: one example per option", "client_region": null, "concurrency": 4, "warmup_calls": 3, "harness_commit": "decidebench@fc49b64"}

Run 2026-10-14. Workload cost is per 1M tasks, not a token tariff.

Synthetic Claude-written items, structural validation and label spot checks. Few-shot examples except explicit zero-shot column. Hosted costs use observed billed usage and dated tariffs. Hosted latency includes provider/gateway, four concurrent calls and three warmups. Luna timing client differs from the other hosted runs, so latency is not a controlled speed ranking.

Checked / reported: 2026-10-17 · Evaluated: 2026-10-14 · Samples: 50

Snapshot: 2026-10-17 · Revision: bbf8f2c77b544952b42be555866729a22c33980d · CC BY 4.0 original dataset; MIT code/report

BFCL formulated as typed decisions; published case exact accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.9664; {"requests": 1694, "answered": 1694, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 1694

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

SGD/SGD-X formulated as typed decisions; published macro-F1 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.4743; {"requests": 2500, "answered": 2500, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 2500

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ContractNLI formulated as typed decisions; published macro-F1 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8339; {"requests": 123, "answered": 123, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 123

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ANLI formulated as typed decisions; published macro-F1 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.7048; {"requests": 3200, "answered": 3200, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 3200

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ToolRet formulated as typed decisions; published nDCG@10 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.673; {"requests": 685, "answered": 685, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 685

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

BPoMP formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8896; {"requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 5000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Humicroedit formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.6355; {"requests": 2628, "answered": 2628, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 2628

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

POP909-CL formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.7975; {"requests": 2000, "answered": 2000, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 2000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

cfcolor formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.5922; {"requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 5000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

MMLU formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8527; {"requests": 14033, "answered": 14033, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 14033

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

GPQA Diamond formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.5153; {"requests": 196, "answered": 196, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 196

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ARC-Easy formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.9861; {"requests": 2376, "answered": 2376, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 2376

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ARC-Challenge formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.9684; {"requests": 1172, "answered": 1172, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 1172

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

WinoGrande formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8611; {"requests": 1267, "answered": 1267, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 1267

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

HellaSwag formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.9589; {"requests": 10042, "answered": 10042, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 10042

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

API-Bank formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8327; {"requests": 508, "answered": 508, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 508

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

GSM8K formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.6683; {"requests": 2638, "answered": 2638, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 2638

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ChessBench formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.2344; {"requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 5000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

MuSR formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.5957; {"requests": 752, "answered": 752, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 752

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

SATA-Bench formulated as typed decisions; published case exact accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.2412; {"requests": 1650, "answered": 1650, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 1650

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

BRIGHT formulated as typed decisions; published nDCG@10 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.4721; {"requests": 220, "answered": 220, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 220

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Amazon ESCI formulated as typed decisions; published macro-F1 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.5459; {"requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 5000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ACOS formulated as typed decisions; published per-review F1 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.2388; {"requests": 1565, "answered": 1565, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 1565

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

FinEntity formulated as typed decisions; published macro-F1 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.9081; {"requests": 979, "answered": 979, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 979

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

BANKING77 formulated as typed decisions; published macro-F1 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8051; {"requests": 3080, "answered": 3080, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 3080

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

iSarcasmEval formulated as typed decisions; published Sarcasm F1 · track A, English for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.6366; {"requests": 4600, "answered": 4600, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 4600

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

VAST formulated as typed decisions; published macro-F1 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.7837; {"requests": 3006, "answered": 3006, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 3006

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

NLI4CT formulated as typed decisions; published macro-F1 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8189; {"requests": 5500, "answered": 5500, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 5500

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

CRUXEval formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8456; {"requests": 570, "answered": 570, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 570

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

CLadder formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8454; {"requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 5000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

HLE formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.1277; {"requests": 501, "answered": 501, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 501

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

CLINC150+OOS formulated as typed decisions; published macro-F1 for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8911; {"requests": 5500, "answered": 5500, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 5500

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Habermas Machine formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.6241; {"requests": 1676, "answered": 1676, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 1676

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

PhishNChips phishing decisions formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.7495; {"requests": 2000, "answered": 2000, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 2000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

MMLU-Pro formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.7284; {"requests": 12032, "answered": 12032, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 12032

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

BBH formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.7835; {"requests": 5507, "answered": 5507, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 5507

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

RAGTruth formulated as typed decisions; published F1 on hallucinated class for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8248; {"requests": 2700, "answered": 2700, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 2700

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

RouterBench formulated as typed decisions; published selected quality (quality objective) for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.792; {"requests": 10000, "answered": 10000, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 10000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

HoVer formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8575; {"requests": 4000, "answered": 4000, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 4000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

When2Call formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.8656; {"requests": 3652, "answered": 3652, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 3652

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

New Yorker formulated as typed decisions; published accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.7765; {"requests": 528, "answered": 528, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 528

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Home appliance simulator formulated as typed decisions; published case exact accuracy for this board population.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw 0.5455; {"requests": 88, "answered": 88, "unsupported": 0, "errors": 0}; coverage/error rules are source-defined.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: 88

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Calibration on the source-defined text sample; confidence alone is not a quality ranking.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw acc: 0.7708; 31 benchmarks. ECE uses the publisher bins and weighting; native Brier convention stays distinct from other studies.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Uses roughly one sixth of requests with discrete correct answers; benchmarks receive equal weight. ToolRet, BRIGHT and RouterBench are excluded. Known training exposure is penalized.

Checked / reported: 2026-10-17 · Samples: 34411

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Calibration on the source-defined text sample; confidence alone is not a quality ranking.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw brier: 0.3168; 31 benchmarks. ECE uses the publisher bins and weighting; native Brier convention stays distinct from other studies.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Uses roughly one sixth of requests with discrete correct answers; benchmarks receive equal weight. ToolRet, BRIGHT and RouterBench are excluded. Known training exposure is penalized.

Checked / reported: 2026-10-17 · Samples: 34411

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Calibration on the source-defined text sample; confidence alone is not a quality ranking.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw conf: 0.7624; 31 benchmarks. ECE uses the publisher bins and weighting; native Brier convention stays distinct from other studies.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Uses roughly one sixth of requests with discrete correct answers; benchmarks receive equal weight. ToolRet, BRIGHT and RouterBench are excluded. Known training exposure is penalized.

Checked / reported: 2026-10-17 · Samples: 34411

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Calibration on the source-defined text sample; confidence alone is not a quality ranking.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Raw ece: 0.0183; 31 benchmarks. ECE uses the publisher bins and weighting; native Brier convention stays distinct from other studies.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Uses roughly one sixth of requests with discrete correct answers; benchmarks receive equal weight. ToolRet, BRIGHT and RouterBench are excluded. Known training exposure is penalized.

Checked / reported: 2026-10-17 · Samples: 34411

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Published Median latency for the board setup; hosted network and local inference are separate.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Local GPU/runtime can differ between submitted and evaluator runs; not isolated architecture speed.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Published p95 latency for the board setup; hosted network and local inference are separate.

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Local GPU/runtime can differ between submitted and evaluator runs; not isolated architecture speed.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Text Decision Index 0.3.2 · Private new-domain score

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Text Decision Index 0.3.2 · Public score

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Text Decision Index 0.3.2 · Private same-skill score

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Text Decision Index 0.3.2 · Full score

Board identity drex-v2.0; author-submitted public run; private tests evaluated by the board. Public model-card serving path; actual revision and precision not otherwise established. Drex public run used unpublished SGLang/router; evaluator checked public Transformers against submitted outputs (98.66% agreement on 831 rows). Training/decontamination details not supplied.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Accuracy as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Accuracy.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Arts as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Arts.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · AUC as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: AUC. Raw AUC 0.984 converted to percent.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Caught @ 5% FP as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Caught @ 5% FP.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Easy as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Easy.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Hard as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Hard.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Knowledge & Reasoning as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Knowledge & Reasoning.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Language as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Language.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Pair accuracy as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Pair accuracy.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Public index as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Public index.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Raw as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Raw.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Retrieval as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Retrieval.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Standard as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Standard.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

Decision Index 0.3.1 (public only) · Tools as reported by Nace.AI

Card nace-ai/drex-v2.0 at 3e6d164cb5c8cf264eec167905cc2448d89ea97d; table Decision Index 0.3.1 (public only).

Published table label: Tools.

Vendor-published observation, distinct from independent and other vendor harnesses. Nace vendor rerun; public-only Decision Index 0.3.1, not the full index. DecideBench few-shot/zero-shot and small JevBench 231-item panel differ from independent leaderboard runs.

Checked / reported: 2026-10-17 · Samples: Not published

Snapshot: 2026-10-06 · Revision: 3e6d164cb5c8cf264eec167905cc2448d89ea97d · Published observations; result reuse terms not specified

OneJev peer figures · Public index

OneJev launch figure, peer-reported 16 October board observation.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17 · Revision: OneJev 1.6 launch · 2026-10-17 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; All-case accuracy.

Named source route nace-ai/drex-v2.0; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 56

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Hard-case accuracy.

Named source route nace-ai/drex-v2.0; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 19

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Public-case accuracy.

Named source route nace-ai/drex-v2.0; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 14

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Hidden-case accuracy.

Named source route nace-ai/drex-v2.0; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 42

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Brier-based confidence score.

Named source route nace-ai/drex-v2.0; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 56

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified