Examenos

Text Decision Index 0.3.2 · Full score

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Text Decision Index 0.3.2 · Public score

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Text Decision Index 0.3.2 · Private same-skill score

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Text Decision Index 0.3.2 · Private new-domain score

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Calibration on the source-defined text sample; confidence alone is not a quality ranking.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Uses roughly one sixth of requests with discrete correct answers; benchmarks receive equal weight. ToolRet, BRIGHT and RouterBench are excluded. Known training exposure is penalized.

Checked / reported: 2026-10-18 · Samples: 34406

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Calibration on the source-defined text sample; confidence alone is not a quality ranking.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Uses roughly one sixth of requests with discrete correct answers; benchmarks receive equal weight. ToolRet, BRIGHT and RouterBench are excluded. Known training exposure is penalized.

Checked / reported: 2026-10-18 · Samples: 34406

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Calibration on the source-defined text sample; confidence alone is not a quality ranking.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Uses roughly one sixth of requests with discrete correct answers; benchmarks receive equal weight. ToolRet, BRIGHT and RouterBench are excluded. Known training exposure is penalized.

Checked / reported: 2026-10-18 · Samples: 34406

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Calibration on the source-defined text sample; confidence alone is not a quality ranking.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access. 32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Uses roughly one sixth of requests with discrete correct answers; benchmarks receive equal weight. ToolRet, BRIGHT and RouterBench are excluded. Known training exposure is penalized.

Checked / reported: 2026-10-18 · Samples: 34406

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Published Median latency for the board setup; hosted network and local inference are separate.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Local board timing on one RTX PRO 6000; not hosted network latency. Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Local GPU/runtime can differ between submitted and evaluator runs; not isolated architecture speed.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Published p95 latency for the board setup; hosted network and local inference are separate.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Local board timing on one RTX PRO 6000; not hosted network latency. Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported. Local GPU/runtime can differ between submitted and evaluator runs; not isolated architecture speed.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

BFCL formulated as typed decisions; published case exact accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "case exact accuracy", "requests": 1694, "answered": 1694, "unsupported": 0, "errors": 0, "median_ms": 73.1}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 1694

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ToolRet formulated as typed decisions; published nDCG@10 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "nDCG@10", "requests": 685, "answered": 685, "unsupported": 0, "errors": 0, "median_ms": 481.6}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 685

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

API-Bank formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 508, "answered": 508, "unsupported": 0, "errors": 0, "median_ms": 187.6}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 508

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

BANKING77 formulated as typed decisions; published macro-F1 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "macro-F1", "requests": 3080, "answered": 3080, "unsupported": 0, "errors": 0, "median_ms": 64.8}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 3080

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

CLINC150+OOS formulated as typed decisions; published macro-F1 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "macro-F1", "requests": 5500, "answered": 5500, "unsupported": 0, "errors": 0, "median_ms": 66.8}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 5500

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

RouterBench formulated as typed decisions; published selected quality (quality objective) for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "selected quality (quality objective)", "requests": 10000, "answered": 10000, "unsupported": 0, "errors": 0, "median_ms": 101.5}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 10000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Home appliance simulator formulated as typed decisions; published case exact accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "case exact accuracy", "requests": 88, "answered": 88, "unsupported": 0, "errors": 0, "median_ms": 1052.7}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 88

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

SGD/SGD-X formulated as typed decisions; published macro-F1 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "macro-F1", "requests": 2500, "answered": 2500, "unsupported": 0, "errors": 0, "median_ms": 66.6}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 2500

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ContractNLI formulated as typed decisions; published macro-F1 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "macro-F1", "requests": 123, "answered": 123, "unsupported": 0, "errors": 0, "median_ms": 1152.6}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 123

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ANLI formulated as typed decisions; published macro-F1 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "macro-F1", "requests": 3200, "answered": 3200, "unsupported": 0, "errors": 0, "median_ms": 61.5}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 3200

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

BPoMP formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 60.4}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 5000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Humicroedit formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 2628, "answered": 2628, "unsupported": 0, "errors": 0, "median_ms": 60.9}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 2628

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

POP909-CL formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 2000, "answered": 2000, "unsupported": 0, "errors": 0, "median_ms": 155.9}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 2000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

cfcolor formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 62}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 5000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

MMLU formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 14033, "answered": 14033, "unsupported": 0, "errors": 0, "median_ms": 60.8}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 14033

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

GPQA Diamond formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 196, "answered": 196, "unsupported": 0, "errors": 0, "median_ms": 61}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 196

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ARC-Easy formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 2376, "answered": 2376, "unsupported": 0, "errors": 0, "median_ms": 59.8}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 2376

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ARC-Challenge formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 1172, "answered": 1172, "unsupported": 0, "errors": 0, "median_ms": 59.8}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 1172

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

WinoGrande formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 1267, "answered": 1267, "unsupported": 0, "errors": 0, "median_ms": 59.6}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 1267

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

HellaSwag formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 10042, "answered": 10042, "unsupported": 0, "errors": 0, "median_ms": 60.8}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 10042

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

GSM8K formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": 0.7301, "metric": "accuracy (4- and 10-choice)", "requests": 2638, "answered": 2638, "unsupported": 0, "errors": 0, "median_ms": null}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 2638

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ChessBench formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 64.5}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 5000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

MuSR formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 752, "answered": 752, "unsupported": 0, "errors": 0, "median_ms": 66}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 752

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

SATA-Bench formulated as typed decisions; published case exact accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "case exact accuracy", "requests": 1650, "answered": 1650, "unsupported": 0, "errors": 0, "median_ms": 87}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 1650

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

BRIGHT formulated as typed decisions; published nDCG@10 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "nDCG@10", "requests": 220, "answered": 220, "unsupported": 0, "errors": 0, "median_ms": 614}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 220

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Amazon ESCI formulated as typed decisions; published macro-F1 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "macro-F1", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 62.7}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 5000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

ACOS formulated as typed decisions; published per-review F1 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "per-review F1", "requests": 1565, "answered": 1565, "unsupported": 0, "errors": 0, "median_ms": 258.7}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 1565

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

FinEntity formulated as typed decisions; published macro-F1 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "macro-F1", "requests": 979, "answered": 979, "unsupported": 0, "errors": 0, "median_ms": 63.8}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 979

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

iSarcasmEval formulated as typed decisions; published Sarcasm F1 · track A, English for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "Sarcasm F1 · track A, English", "requests": 4600, "answered": 4600, "unsupported": 0, "errors": 0, "median_ms": 61.4}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 4600

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

VAST formulated as typed decisions; published macro-F1 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "macro-F1", "requests": 3006, "answered": 3006, "unsupported": 0, "errors": 0, "median_ms": 61.2}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 3006

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

NLI4CT formulated as typed decisions; published macro-F1 for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "macro-F1", "requests": 5500, "answered": 5500, "unsupported": 0, "errors": 0, "median_ms": 63.4}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 5500

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

CRUXEval formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 570, "answered": 570, "unsupported": 0, "errors": 0, "median_ms": 61}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 570

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

CLadder formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 5000, "answered": 5000, "unsupported": 0, "errors": 0, "median_ms": 61.1}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 5000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

HLE formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 501, "answered": 501, "unsupported": 0, "errors": 0, "median_ms": 61.8}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 501

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Habermas Machine formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 1676, "answered": 1676, "unsupported": 0, "errors": 0, "median_ms": 65.1}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 1676

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

PhishNChips phishing decisions formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 2000, "answered": 2000, "unsupported": 0, "errors": 0, "median_ms": 77.1}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 2000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

MMLU-Pro formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 12032, "answered": 12032, "unsupported": 0, "errors": 0, "median_ms": 61.5}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 12032

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

BBH formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 5507, "answered": 5507, "unsupported": 0, "errors": 0, "median_ms": 61}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 5507

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

RAGTruth formulated as typed decisions; published F1 on hallucinated class for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "F1 on hallucinated class", "requests": 2700, "answered": 2700, "unsupported": 0, "errors": 0, "median_ms": 64.8}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 2700

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

HoVer formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 4000, "answered": 4000, "unsupported": 0, "errors": 0, "median_ms": 62.9}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 4000

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

When2Call formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 3652, "answered": 3652, "unsupported": 0, "errors": 0, "median_ms": 64.2}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 3652

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

New Yorker formulated as typed decisions; published accuracy for this board population.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Exact published per-benchmark observation; no composite reconstructed. Counts/errors and timing: {"jev": null, "metric": "accuracy", "requests": 528, "answered": 528, "unsupported": 0, "errors": 0, "median_ms": 61}

Community Decision Index 0.3.2; distinct from AA. Full = 20% public + 50% private same-skills + 30% private new-domains on publisher-normalized scales. Public suite is chance-corrected and coverage-adjusted; unanswered counts wrong. Some public runs are author-submitted and private runs evaluator-run; each record preserves that status. Small differences can be tied by the publisher. Snapshot date is not execution date. No underlying private test text is imported.

Checked / reported: 2026-10-18 · Samples: 528

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 11

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 11

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 11

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 9

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 9

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 9

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 6

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 6

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 6

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 5

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 5

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 5

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 6

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 6

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Source-published category quantity; skill-adjusted, raw and combined values are distinct.

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

32 benchmarks with a per-field right or wrong answer (ToolRet, BRIGHT and RouterBench excluded), each weighted equally; a 1-in-6 sample of requests; confidence is the probability placed on the chosen option. Rows an entrant trained on count as wrong here too.

Checked / reported: 2026-10-18 · Samples: 6

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

HF text 0.3.2 · Raw balanced index

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

Source-published alternative aggregate, separate from Full and Public scores; no Examenos recomputation.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

HF text 0.3.2 · Public breadth index

Board exact pplx-decider-flash-v1.2. Public results submitted by Perplexity; private tests run by the board on one RTX PRO 6000. One question per request, author serving code. Exact execution SHA/precision not restated by this source.

Submitted directly as a results dataset (perplexity-ai/decision-index-results), no PR. Submitted public 0.3 run adopted (kit rescore 60.06, equal to the claim). Private tests run by the lab on one RTX PRO 6000 with the author's serving; decision agreement with the submitted rows 99.58% on 827 rows (bar 98.5%). Distilled from the Perplexity Decider v1.2 27B models; the checkpoint is a weight average of several runs (author-declared). One question per request, the submitter's chosen mode. Model repository is private. Current official HF API is public/ungated; the board’s historical private-repository note describes its source snapshot, not current access.

Source-published alternative aggregate, separate from Full and Public scores; no Examenos recomputation.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T16:39:08Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Multimodal typed-choice vision evaluation · Full

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Public and private panels are separate; Full is the board-published aggregate, not vendor card scores.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP)

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Multimodal typed-choice vision evaluation · Public

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Public and private panels are separate; Full is the board-published aggregate, not vendor card scores.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP)

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Multimodal typed-choice vision evaluation · Private

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Public and private panels are separate; Full is the board-published aggregate, not vendor card scores.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP)

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on CV-Bench; pub population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 2038.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on CV-Bench; private population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 2038.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on BLINK; pub population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 961.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on BLINK; private population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 961.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on RealWorldQA; pub population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 589.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on RealWorldQA; private population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 589.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on CharXiv; pub population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 532.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on CharXiv; private population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 532.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on InfographicVQA; pub population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 244.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on InfographicVQA; private population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 244.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on Mind2Web; pub population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 973.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on Mind2Web; private population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 973.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on Winoground; pub population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 800.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on Winoground; private population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 800.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on KIE (CORD+FUNSD); pub population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 2465.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on KIE (CORD+FUNSD); private population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 2465.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on Moderation (Hateful Memes); pub population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 1213.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on Moderation (Hateful Memes); private population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 1213.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on R-Bench-M; pub population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 665.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

Typed vision decisions on MMMU-Pro vision; pub population.

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Native board score; private normalized values can be negative. No clipping or substituted subset.

Vision board 0.3.2, 50% public and 50% private; public/private are distinct from the text board. Source benchmark weights and image caps apply. author cap 1,048,576 px lifted to the board cap via the documented media_kwargs (processor max 12.8 MP) Private normalized scores may be below zero; no clipping. Public sample count: 1730.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published observations; result reuse terms not specified

HF vision 0.3.2 · Chosen-answer accuracy

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Source-defined vision calibration population and confidence convention; distinct from text calibration.

Checked / reported: 2026-10-18 · Samples: 13695

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

HF vision 0.3.2 · Mean chosen confidence

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Source-defined vision calibration population and confidence convention; distinct from text calibration.

Checked / reported: 2026-10-18 · Samples: 13695

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

HF vision 0.3.2 · Chosen-answer ECE

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Source-defined vision calibration population and confidence convention; distinct from text calibration.

Checked / reported: 2026-10-18 · Samples: 13695

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

HF vision 0.3.2 · Incorrect and >95% confident

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Source-defined vision calibration population and confidence convention; distinct from text calibration.

Checked / reported: 2026-10-18 · Samples: 13695

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

HF vision 0.3.2 · Time per row (source)

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Published resource observation for this suite and hardware; not API speed or billing. Detailed distribution statistic for time-per-row is not inferred.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

HF vision 0.3.2 · Evaluation GPU hours

Exact vision entrant pplx-decider-flash-v1.2. Ran through the checkpoint's own pplx_decider engine (the submitted text adapter), which accepts images. Effective image budget 1.6 MP. author cap equals the board cap (decision_config.json max_pixels)

Published resource observation for this suite and hardware; not API speed or billing. Detailed distribution statistic for time-per-row is not inferred.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16T19:00:13Z · Revision: 0ddf6758073eb77ae449a3541f57286d7f39d93f · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · joint · Knowledge

Perplexity card; joint request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · single · Knowledge

Perplexity card; single request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · joint · Language

Perplexity card; joint request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · single · Language

Perplexity card; single request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · joint · Retrieval

Perplexity card; joint request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · single · Retrieval

Perplexity card; single request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · joint · Tools

Perplexity card; joint request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · single · Tools

Perplexity card; single request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · joint · Arts

Perplexity card; joint request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · single · Arts

Perplexity card; single request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · joint · Overall Decision Index

Perplexity card; joint request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 public · single · Overall Decision Index

Perplexity card; single request mode; original published suite weights. Flash distillation from 27B probability distributions.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 · Vision public index

Vendor vision public run; image cap 1,638,400 pixels.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations. Vendor card values differ slightly from the board rerun and are preserved separately.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 joint minus single · SATA-Bench (select all that apply)

Joint requests versus one question per request; one per option.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Signed task-score difference, not absolute task accuracy. Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 joint minus single · Home appliance simulator

Joint requests versus one question per request; steps of one task.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Signed task-score difference, not absolute task accuracy. Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 joint minus single · FinEntity

Joint requests versus one question per request; one per entity of a text.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Signed task-score difference, not absolute task accuracy. Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 joint minus single · PhishNChips

Joint requests versus one question per request; facets of one message.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Signed task-score difference, not absolute task accuracy. Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 joint minus single · BRIGHT (reasoning-intensive retrieval)

Joint requests versus one question per request; one per candidate document.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Signed task-score difference, not absolute task accuracy. Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 joint minus single · ToolRet (tool retrieval)

Joint requests versus one question per request; one per candidate tool.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Signed task-score difference, not absolute task accuracy. Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 joint minus single · BFCL (function calling)

Joint requests versus one question per request; one per candidate call.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Signed task-score difference, not absolute task accuracy. Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity v1.2 joint minus single · ACOS (aspect-category sentiment)

Joint requests versus one question per request; one per category.

Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Signed task-score difference, not absolute task accuracy. Vendor-published measurements. Decision Index 0.3 public suite, not AA, evaluator-private Full, or a controlled cross-provider comparison. Official training splits were included with test decontamination; held-out claims are limited to their stated populations.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity dev · calibrated · Accuracy

Joint request format; calibrated probabilities; fitted temperature 1.554.

1,497-question development sample, separate from temperature-fitting data. Native Brier and ECE pp remain distinct.

Checked / reported: 2026-10-18 · Samples: 1497

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity dev · calibrated · ECE

Joint request format; calibrated probabilities; fitted temperature 1.554.

1,497-question development sample, separate from temperature-fitting data. Native Brier and ECE pp remain distinct.

Checked / reported: 2026-10-18 · Samples: 1497

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity dev · uncalibrated · ECE

Joint request format; uncalibrated probabilities; fitted temperature 1.554.

1,497-question development sample, separate from temperature-fitting data. Native Brier and ECE pp remain distinct.

Checked / reported: 2026-10-18 · Samples: 1497

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity dev · calibrated · Brier

Joint request format; calibrated probabilities; fitted temperature 1.554.

1,497-question development sample, separate from temperature-fitting data. Native Brier and ECE pp remain distinct.

Checked / reported: 2026-10-18 · Samples: 1497

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity dev · uncalibrated · Brier

Joint request format; uncalibrated probabilities; fitted temperature 1.554.

1,497-question development sample, separate from temperature-fitting data. Native Brier and ECE pp remain distinct.

Checked / reported: 2026-10-18 · Samples: 1497

Snapshot: 2026-10-16 · Revision: Official v1.2 cards · 2026-10-16 · Published aggregate observations; result reuse terms not separately specified

Perplexity 243K log spot check · reference · Approximate peak gpu memory

Vendor long-context spot check; reference path.

Published approximate workload observation; no inference of provider latency or generic scaling law.

One fact retrieval from a 243K-token log on one GB200. Vendor spot check only; not validated whole-context accuracy. P(correct)>0.999 is a bound, not an exact point result.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: 225c913ddc8761a8264f04f64548e1f97e443eb3 · Published aggregate observations; result reuse terms not separately specified

Perplexity 243K log spot check · reference · Approximate request duration

Vendor long-context spot check; reference path.

Published approximate workload observation; no inference of provider latency or generic scaling law.

One fact retrieval from a 243K-token log on one GB200. Vendor spot check only; not validated whole-context accuracy. P(correct)>0.999 is a bound, not an exact point result.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: 225c913ddc8761a8264f04f64548e1f97e443eb3 · Published aggregate observations; result reuse terms not separately specified

Perplexity 243K log spot check · fast · Approximate peak gpu memory

Vendor long-context spot check; fast path.

Published approximate workload observation; no inference of provider latency or generic scaling law.

One fact retrieval from a 243K-token log on one GB200. Vendor spot check only; not validated whole-context accuracy. P(correct)>0.999 is a bound, not an exact point result.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: 225c913ddc8761a8264f04f64548e1f97e443eb3 · Published aggregate observations; result reuse terms not separately specified

Perplexity 243K log spot check · fast · Approximate request duration

Vendor long-context spot check; fast path.

Published approximate workload observation; no inference of provider latency or generic scaling law.

One fact retrieval from a 243K-token log on one GB200. Vendor spot check only; not validated whole-context accuracy. P(correct)>0.999 is a bound, not an exact point result.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-16 · Revision: 225c913ddc8761a8264f04f64548e1f97e443eb3 · Published aggregate observations; result reuse terms not separately specified

OneJev peer figures · Public index

OneJev launch figure, peer-reported 16 October board observation.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17 · Revision: OneJev 1.6 launch · 2026-10-17 · Published aggregate observations; result reuse terms not separately specified

OneJev peer figures · Public Knowledge

OneJev launch figure, peer-reported 16 October board observation.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17 · Revision: OneJev 1.6 launch · 2026-10-17 · Published aggregate observations; result reuse terms not separately specified

OneJev peer figures · Public Language

OneJev launch figure, peer-reported 16 October board observation.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17 · Revision: OneJev 1.6 launch · 2026-10-17 · Published aggregate observations; result reuse terms not separately specified

OneJev peer figures · Public Retrieval

OneJev launch figure, peer-reported 16 October board observation.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17 · Revision: OneJev 1.6 launch · 2026-10-17 · Published aggregate observations; result reuse terms not separately specified

OneJev peer figures · Public Tools

OneJev launch figure, peer-reported 16 October board observation.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17 · Revision: OneJev 1.6 launch · 2026-10-17 · Published aggregate observations; result reuse terms not separately specified

OneJev peer figures · Public Arts

OneJev launch figure, peer-reported 16 October board observation.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Vendor-run Decision Index 0.3 public kit; 37 benchmarks / 140,178 requests as reported. Source quotes peers from the 16 October board. Not AA, evaluator-private Full, or S1MB. Exact execution dates, budgets and individual run variances are not published.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17 · Revision: OneJev 1.6 launch · 2026-10-17 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/aegis2 / test · binary_brier. Population 100 cases / 148 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.13739585604750978,"accuracy":0.7905405405405406,"positive_recall":0.704225352112676,"specificity":0.8701298701298701,"precision":0.8333333333333334,"balanced_accuracy":0.787177611121273,"f1":0.7633587786259542,"true_positive":50,"false_positive":10,"false_negative":21,"true_negative":67,"positive_prevalence":0.4797297297297297,"majority_accuracy_baseline":0.5202702702702703,"constant_brier_baseline":0.24958911614317017,"brier_skill":0.44951182899859987,"baseline_adjusted_skill":0.5743552222425461,"baseline_adjusted_score":0.5743552222425461}. Counts: {"cases": 100, "expected": 148, "succeeded": 148, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/aqua_rat / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.54,"uniform_accuracy_baseline":0.2005,"fixed_answer_accuracy_baseline":0.29,"baseline_adjusted_skill":0.3521126760563381,"baseline_adjusted_score":0.3521126760563381}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/arc / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.98,"uniform_accuracy_baseline":0.2533333333333333,"fixed_answer_accuracy_baseline":0.32,"baseline_adjusted_skill":0.9705882352941176,"baseline_adjusted_score":0.9705882352941176}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/arct / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.53,"baseline_adjusted_skill":0.7872340425531915,"baseline_adjusted_score":0.7872340425531915}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/argument_quality / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.98,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.51,"baseline_adjusted_skill":0.9591836734693877,"baseline_adjusted_score":0.9591836734693877}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/argument_quality / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.07983197841966329,"accuracy":0.59170011514,"positive_recall":0.7205236606630496,"specificity":0.3850160313595339,"precision":0.6527456222058823,"balanced_accuracy":0.5527698460112918,"f1":0.6849620406727625,"true_positive":44.38670231,"false_positive":23.61329769,"false_negative":17.216690795999998,"true_negative":14.783309204,"positive_prevalence":0.61603393106,"majority_accuracy_baseline":0.61603393106,"constant_brier_baseline":0.07908707655202402,"brier_skill":-0.009418755884209151,"baseline_adjusted_skill":0.10553969202258351,"baseline_adjusted_score":0.10553969202258351}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/babi_nli / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.05746965271455907,"accuracy":0.9,"positive_recall":0.92,"specificity":0.88,"precision":0.8846153846153846,"balanced_accuracy":0.9,"f1":0.9019607843137255,"true_positive":46,"false_positive":6,"false_negative":4,"true_negative":44,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.7701213891417638,"baseline_adjusted_skill":0.8,"baseline_adjusted_score":0.8}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/banking77 / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.88,"uniform_accuracy_baseline":0.012987012987012986,"fixed_answer_accuracy_baseline":0.05,"baseline_adjusted_skill":0.8736842105263158,"baseline_adjusted_score":0.8736842105263158}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/bbq / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.97,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.4,"baseline_adjusted_skill":0.95,"baseline_adjusted_score":0.95}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/boardgameqa / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.83,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.36,"baseline_adjusted_skill":0.7343749999999999,"baseline_adjusted_score":0.7343749999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/canttalk / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.155809542217997,"accuracy":0.79,"positive_recall":1,"specificity":0.58,"precision":0.704225352112676,"balanced_accuracy":0.79,"f1":0.8264462809917356,"true_positive":50,"false_positive":21,"false_negative":0,"true_negative":29,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.376761831128012,"baseline_adjusted_skill":0.5800000000000001,"baseline_adjusted_score":0.5800000000000001}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/civil_comments / test · binary_brier. Population 100 cases / 700 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.08335498275222239,"accuracy":0.6881045465244513,"positive_recall":0.737583279395952,"specificity":0.6650654320400565,"precision":0.50627323900016,"balanced_accuracy":0.7013243557180042,"f1":0.600420820841382,"true_positive":164.03252943605185,"false_positive":159.96747056394815,"false_negative":58.35934686893597,"true_negative":317.640653131064,"positive_prevalence":0.3177026804356969,"majority_accuracy_baseline":0.6822973195643032,"constant_brier_baseline":0.08872692397768642,"brier_skill":0.06054465752486804,"baseline_adjusted_skill":0.40264871143600844,"baseline_adjusted_score":0.40264871143600844}. Counts: {"cases": 100, "expected": 700, "succeeded": 700, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/cladder / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.11574067667846258,"accuracy":0.8,"positive_recall":0.78,"specificity":0.82,"precision":0.8125,"balanced_accuracy":0.8,"f1":0.7959183673469388,"true_positive":39,"false_positive":9,"false_negative":11,"true_negative":41,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.5370372932861497,"baseline_adjusted_skill":0.6000000000000001,"baseline_adjusted_score":0.6000000000000001}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/clinc / test · target_mass_at_prediction. Population 151 cases / 151 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9536423841059603,"uniform_accuracy_baseline":0.006622516556291391,"fixed_answer_accuracy_baseline":0.039735099337748346,"baseline_adjusted_skill":0.9517241379310345,"baseline_adjusted_score":0.9517241379310345}. Counts: {"cases": 151, "expected": 151, "succeeded": 151, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 151

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/commonsensemoralja-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.0714469139758872,"accuracy":0.91,"positive_recall":0.88,"specificity":0.94,"precision":0.9361702127659575,"balanced_accuracy":0.9099999999999999,"f1":0.9072164948453608,"true_positive":132,"false_positive":9,"false_negative":18,"true_negative":141,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.7142123440964512,"baseline_adjusted_skill":0.8199999999999998,"baseline_adjusted_score":0.8199999999999998}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/contract_nli / test · target_mass_at_prediction. Population 100 cases / 1700 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.87,"uniform_accuracy_baseline":0.3333333333333333,"fixed_answer_accuracy_baseline":0.4576470588235294,"baseline_adjusted_skill":0.7603036876355749,"baseline_adjusted_score":0.7603036876355749}. Counts: {"cases": 100, "expected": 1700, "succeeded": 1700, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/corr2cause / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.76,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.4,"baseline_adjusted_skill":0.6,"baseline_adjusted_score":0.6}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/creak / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.07905205591488353,"accuracy":0.88,"positive_recall":0.94,"specificity":0.82,"precision":0.8392857142857143,"balanced_accuracy":0.8799999999999999,"f1":0.8867924528301887,"true_positive":47,"false_positive":9,"false_negative":3,"true_negative":41,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.683791776340466,"baseline_adjusted_skill":0.7599999999999998,"baseline_adjusted_score":0.7599999999999998}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/crows_pairs / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.8,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.52,"baseline_adjusted_skill":0.5833333333333334,"baseline_adjusted_score":0.5833333333333334}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/cvbench-depth-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9366666666666666,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.5,"baseline_adjusted_skill":0.8733333333333333,"baseline_adjusted_score":0.8733333333333333}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/cvbench-distance-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.88,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.52,"baseline_adjusted_skill":0.75,"baseline_adjusted_score":0.75}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/cvbench-relation-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9633333333333334,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.5266666666666666,"baseline_adjusted_skill":0.9225352112676057,"baseline_adjusted_score":0.9225352112676057}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/dbpedia / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.99,"uniform_accuracy_baseline":0.07142857142857142,"fixed_answer_accuracy_baseline":0.11,"baseline_adjusted_skill":0.9887640449438202,"baseline_adjusted_score":0.9887640449438202}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/defeasible_nli / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.86,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.54,"baseline_adjusted_skill":0.6956521739130435,"baseline_adjusted_score":0.6956521739130435}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/esci / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.42,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.27,"baseline_adjusted_skill":0.20547945205479448,"baseline_adjusted_score":0.20547945205479448}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/ethics / test · target_mass_at_prediction. Population 18 cases / 18 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.8333333333333334,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.6666666666666666,"baseline_adjusted_skill":0.5000000000000001,"baseline_adjusted_score":0.5000000000000001}. Counts: {"cases": 18, "expected": 18, "succeeded": 18, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 18

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/ethics / test · binary_brier. Population 82 cases / 82 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.15892694468025784,"accuracy":0.8048780487804879,"positive_recall":0.775,"specificity":0.8333333333333334,"precision":0.8157894736842105,"balanced_accuracy":0.8041666666666667,"f1":0.7948717948717948,"true_positive":31,"false_positive":7,"false_negative":9,"true_negative":35,"positive_prevalence":0.4878048780487805,"majority_accuracy_baseline":0.5121951219512195,"constant_brier_baseline":0.2498512790005949,"brier_skill":0.3639138237916347,"baseline_adjusted_skill":0.6083333333333334,"baseline_adjusted_score":0.6083333333333334}. Counts: {"cases": 82, "expected": 82, "succeeded": 82, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 82

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/ethos / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.12228085154572353,"accuracy":0.83,"positive_recall":0.82,"specificity":0.84,"precision":0.8367346938775511,"balanced_accuracy":0.83,"f1":0.8282828282828283,"true_positive":41,"false_positive":8,"false_negative":9,"true_negative":42,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.5108765938171058,"baseline_adjusted_skill":0.6599999999999999,"baseline_adjusted_score":0.6599999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/few_nerd / test · target_mass_at_prediction. Population 100 cases / 358 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.7960893854748603,"uniform_accuracy_baseline":0.125,"fixed_answer_accuracy_baseline":0.2206703910614525,"baseline_adjusted_skill":0.7383512544802867,"baseline_adjusted_score":0.7383512544802867}. Counts: {"cases": 100, "expected": 358, "succeeded": 358, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/fol_nli / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.7,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.39,"baseline_adjusted_skill":0.5081967213114753,"baseline_adjusted_score":0.5081967213114753}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/followir_core17 / test · binary_brier. Population 100 cases / 200 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.15751198773962255,"accuracy":0.775,"positive_recall":0.4090909090909091,"specificity":0.8782051282051282,"precision":0.4864864864864865,"balanced_accuracy":0.6436480186480187,"f1":0.4444444444444444,"true_positive":18,"false_positive":19,"false_negative":26,"true_negative":137,"positive_prevalence":0.22,"majority_accuracy_baseline":0.78,"constant_brier_baseline":0.1716,"brier_skill":0.08209797354532311,"baseline_adjusted_skill":0.28729603729603737,"baseline_adjusted_score":0.28729603729603737}. Counts: {"cases": 100, "expected": 200, "succeeded": 200, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/followir_news21 / test · binary_brier. Population 100 cases / 200 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.16994697531300815,"accuracy":0.755,"positive_recall":0.18,"specificity":0.9466666666666667,"precision":0.5294117647058824,"balanced_accuracy":0.5633333333333334,"f1":0.26865671641791045,"true_positive":9,"false_positive":8,"false_negative":41,"true_negative":142,"positive_prevalence":0.25,"majority_accuracy_baseline":0.75,"constant_brier_baseline":0.1875,"brier_skill":0.09361613166395655,"baseline_adjusted_skill":0.1266666666666667,"baseline_adjusted_score":0.1266666666666667}. Counts: {"cases": 100, "expected": 200, "succeeded": 200, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/followir_robust04 / test · binary_brier. Population 100 cases / 200 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.03220024481798704,"accuracy":0.97,"positive_recall":0.8421052631578947,"specificity":0.9834254143646409,"precision":0.8421052631578947,"balanced_accuracy":0.9127653387612678,"f1":0.8421052631578947,"true_positive":16,"false_positive":3,"false_negative":3,"true_negative":178,"positive_prevalence":0.095,"majority_accuracy_baseline":0.905,"constant_brier_baseline":0.085975,"brier_skill":0.6254696735331544,"baseline_adjusted_skill":0.8255306775225355,"baseline_adjusted_score":0.8255306775225355}. Counts: {"cases": 100, "expected": 200, "succeeded": 200, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-japanese-diverse-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.99,"uniform_accuracy_baseline":0.3333333333333333,"fixed_answer_accuracy_baseline":0.3333333333333333,"baseline_adjusted_skill":0.985,"baseline_adjusted_score":0.985}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-japanese-diverse-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.012181035705602956,"accuracy":0.9866666666666667,"positive_recall":0.9933333333333333,"specificity":0.98,"precision":0.9802631578947368,"balanced_accuracy":0.9866666666666666,"f1":0.9867549668874173,"true_positive":149,"false_positive":3,"false_negative":1,"true_negative":147,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.9512758571775882,"baseline_adjusted_skill":0.9733333333333332,"baseline_adjusted_score":0.9733333333333332}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-japanese-knowledge-hard-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9666666666666667,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.25,"baseline_adjusted_skill":0.9555555555555556,"baseline_adjusted_score":0.9555555555555556}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-japanese-knowledge-hard-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.13597030963270368,"accuracy":0.8033333333333333,"positive_recall":0.9733333333333334,"specificity":0.6333333333333333,"precision":0.7263681592039801,"balanced_accuracy":0.8033333333333333,"f1":0.8319088319088319,"true_positive":146,"false_positive":55,"false_negative":4,"true_negative":95,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.4561187614691853,"baseline_adjusted_skill":0.6066666666666667,"baseline_adjusted_score":0.6066666666666667}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-japanese-knowledge-hard-score / test · normalized_expected_score_mae. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.13888804177217165,"normalized_score_rmse":0.20206235392789978,"constant_score_baseline":0.5,"constant_mae_baseline":0.3333333333333333,"baseline_adjusted_skill":0.583335874683485,"baseline_adjusted_score":0.583335874683485}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-japanese-math-formal-hard-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.47333333333333333,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.25,"baseline_adjusted_skill":0.29777777777777775,"baseline_adjusted_score":0.29777777777777775}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-japanese-math-formal-hard-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.25041277066168544,"accuracy":0.5533333333333333,"positive_recall":0.3,"specificity":0.8066666666666666,"precision":0.6081081081081081,"balanced_accuracy":0.5533333333333333,"f1":0.4017857142857143,"true_positive":45,"false_positive":29,"false_negative":105,"true_negative":121,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":-0.0016510826467417417,"baseline_adjusted_skill":0.10666666666666669,"baseline_adjusted_score":0.10666666666666669}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-japanese-math-formal-hard-score / test · normalized_expected_score_mae. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.31805233240696024,"normalized_score_rmse":0.37854095996952997,"constant_score_baseline":0.5,"constant_mae_baseline":0.3,"baseline_adjusted_skill":-0.06017444135653416,"baseline_adjusted_score":0}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-vision-easy-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9766666666666667,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.3,"baseline_adjusted_skill":0.9666666666666669,"baseline_adjusted_score":0.9666666666666669}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-vision-easy-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.026966930627868056,"accuracy":0.9666666666666667,"positive_recall":0.9664429530201343,"specificity":0.9668874172185431,"precision":0.9664429530201343,"balanced_accuracy":0.9666651851193386,"f1":0.9664429530201343,"true_positive":144,"false_positive":5,"false_negative":5,"true_negative":146,"positive_prevalence":0.49666666666666665,"majority_accuracy_baseline":0.5033333333333334,"constant_brier_baseline":0.2499888888888889,"brier_skill":0.8921274831544458,"baseline_adjusted_skill":0.9333303702386773,"baseline_adjusted_score":0.9333303702386773}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-vision-easy-score / test · normalized_expected_score_mae. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.16900248995641187,"normalized_score_rmse":0.2177593672052418,"constant_score_baseline":0.5,"constant_mae_baseline":0.36666666666666664,"baseline_adjusted_skill":0.5390841183006949,"baseline_adjusted_score":0.5390841183006949}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-vision-hard-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.7166666666666667,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.2866666666666667,"baseline_adjusted_skill":0.6028037383177569,"baseline_adjusted_score":0.6028037383177569}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-vision-hard-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.16074834557130352,"accuracy":0.76,"positive_recall":0.8407643312101911,"specificity":0.6713286713286714,"precision":0.7374301675977654,"balanced_accuracy":0.7560465012694313,"f1":0.7857142857142857,"true_positive":132,"false_positive":47,"false_negative":25,"true_negative":96,"positive_prevalence":0.5233333333333333,"majority_accuracy_baseline":0.5233333333333333,"constant_brier_baseline":0.24945555555555557,"brier_skill":0.35560326482484894,"baseline_adjusted_skill":0.5120930025388626,"baseline_adjusted_score":0.5120930025388626}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/generalization-vision-hard-score / test · normalized_expected_score_mae. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.23466040064362403,"normalized_score_rmse":0.2850127992826398,"constant_score_baseline":0.5,"constant_mae_baseline":0.3016666666666667,"baseline_adjusted_skill":0.22212021886091482,"baseline_adjusted_score":0.22212021886091482}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/go_emotions / test · binary_brier. Population 100 cases / 2800 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.09584169025468896,"accuracy":0.8305535714285714,"positive_recall":0.7456601527626228,"specificity":0.8359728721686435,"precision":0.22492519449431478,"balanced_accuracy":0.7908165124656331,"f1":0.34560125054596447,"true_positive":125.28333333333333,"false_positive":431.71666666666664,"false_negative":42.733333333333334,"true_negative":2200.266666666667,"positive_prevalence":0.060005952380952375,"majority_accuracy_baseline":0.9399940476190476,"constant_brier_baseline":0.03220097218679138,"brier_skill":-1.976360145238179,"baseline_adjusted_skill":0.5816330249312662,"baseline_adjusted_score":0.5816330249312662}. Counts: {"cases": 100, "expected": 2800, "succeeded": 2800, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/gpqa-ja-main-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.42333333333333334,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.25,"baseline_adjusted_skill":0.23111111111111113,"baseline_adjusted_score":0.23111111111111113}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/gretel_pii / test · target_mass_at_prediction. Population 100 cases / 353 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.8101983002832861,"uniform_accuracy_baseline":0.027777777777777776,"fixed_answer_accuracy_baseline":0.1274787535410765,"baseline_adjusted_skill":0.7824675324675324,"baseline_adjusted_score":0.7824675324675324}. Counts: {"cases": 100, "expected": 353, "succeeded": 353, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/gsm8k / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.79,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.3,"baseline_adjusted_skill":0.7000000000000001,"baseline_adjusted_score":0.7000000000000001}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/hans / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.05729234225934866,"accuracy":0.9,"positive_recall":1,"specificity":0.8,"precision":0.8333333333333334,"balanced_accuracy":0.9,"f1":0.9090909090909091,"true_positive":50,"false_positive":10,"false_negative":0,"true_negative":40,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.7708306309626054,"baseline_adjusted_skill":0.8,"baseline_adjusted_score":0.8}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/hatecheck / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.03663836439467515,"accuracy":0.96,"positive_recall":1,"specificity":0.92,"precision":0.9259259259259259,"balanced_accuracy":0.96,"f1":0.9615384615384616,"true_positive":50,"false_positive":4,"false_negative":0,"true_negative":46,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.8534465424212994,"baseline_adjusted_skill":0.9199999999999999,"baseline_adjusted_score":0.9199999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/hh_rlhf / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.64,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.61,"baseline_adjusted_skill":0.07692307692307698,"baseline_adjusted_score":0.07692307692307698}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/hwu64 / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9,"uniform_accuracy_baseline":0.015625,"fixed_answer_accuracy_baseline":0.06,"baseline_adjusted_skill":0.8936170212765959,"baseline_adjusted_score":0.8936170212765959}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/impli / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.05932013379977822,"accuracy":0.92,"positive_recall":0.94,"specificity":0.9,"precision":0.9038461538461539,"balanced_accuracy":0.9199999999999999,"f1":0.9215686274509803,"true_positive":47,"false_positive":5,"false_negative":3,"true_negative":45,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.7627194648008871,"baseline_adjusted_skill":0.8399999999999999,"baseline_adjusted_score":0.8399999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/jamc-qa-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.42,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.25,"baseline_adjusted_skill":0.22666666666666666,"baseline_adjusted_score":0.22666666666666666}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/jamp-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.6833333333333333,"uniform_accuracy_baseline":0.3333333333333333,"fixed_answer_accuracy_baseline":0.3333333333333333,"baseline_adjusted_skill":0.525,"baseline_adjusted_score":0.525}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/janli-choice / test · target_mass_at_prediction. Population 285 cases / 285 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.7508771929824561,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.5263157894736842,"baseline_adjusted_skill":0.4740740740740741,"baseline_adjusted_score":0.4740740740740741}. Counts: {"cases": 285, "expected": 285, "succeeded": 285, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 285

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/jcola-noul / test · binary_brier. Population 299 cases / 299 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.2570381341696935,"accuracy":0.5886287625418061,"positive_recall":0.9333333333333333,"specificity":0.24161073825503357,"precision":0.5533596837944664,"balanced_accuracy":0.5874720357941835,"f1":0.6947890818858561,"true_positive":140,"false_positive":113,"false_negative":10,"true_negative":36,"positive_prevalence":0.5016722408026756,"majority_accuracy_baseline":0.5016722408026756,"constant_brier_baseline":0.24999720361069788,"brier_skill":-0.028164037266432684,"baseline_adjusted_skill":0.174944071588367,"baseline_adjusted_score":0.174944071588367}. Counts: {"cases": 299, "expected": 299, "succeeded": 299, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 299

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/jcommonsenseqa-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9366666666666666,"uniform_accuracy_baseline":0.2,"fixed_answer_accuracy_baseline":0.2,"baseline_adjusted_skill":0.9208333333333332,"baseline_adjusted_score":0.9208333333333332}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/jculture-mcq-noul / test · binary_brier. Population 283 cases / 283 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.3783705189800387,"accuracy":0.5265017667844523,"positive_recall":0.9022556390977443,"specificity":0.19333333333333333,"precision":0.4979253112033195,"balanced_accuracy":0.5477944862155388,"f1":0.6417112299465241,"true_positive":120,"false_positive":121,"false_negative":13,"true_negative":29,"positive_prevalence":0.46996466431095407,"majority_accuracy_baseline":0.5300353356890459,"constant_brier_baseline":0.24909787861004634,"brier_skill":-0.5189632328116449,"baseline_adjusted_skill":0.09558897243107767,"baseline_adjusted_score":0.09558897243107767}. Counts: {"cases": 283, "expected": 283, "succeeded": 283, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 283

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/jmmlu-japan-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.7866666666666666,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.25,"baseline_adjusted_skill":0.7155555555555555,"baseline_adjusted_score":0.7155555555555555}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/jmteb-retrieval / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.96,"uniform_accuracy_baseline":0.20616666666666666,"fixed_answer_accuracy_baseline":0.21333333333333335,"baseline_adjusted_skill":0.9491525423728813,"baseline_adjusted_score":0.9491525423728813}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/jnli-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.8633333333333333,"uniform_accuracy_baseline":0.3333333333333333,"fixed_answer_accuracy_baseline":0.3333333333333333,"baseline_adjusted_skill":0.7949999999999999,"baseline_adjusted_score":0.7949999999999999}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/jsem-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.6433333333333333,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.37333333333333335,"baseline_adjusted_skill":0.43085106382978716,"baseline_adjusted_score":0.43085106382978716}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/jsick-sts-score / test · normalized_expected_score_mae. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.1319941185813407,"normalized_score_rmse":0.15931710484046746,"constant_score_baseline":0.5,"constant_mae_baseline":0.26208333333333333,"baseline_adjusted_skill":0.4963658432508463,"baseline_adjusted_score":0.4963658432508463}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/jsts-score / test · normalized_expected_score_mae. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.09652835165272654,"normalized_score_rmse":0.11960000061302878,"constant_score_baseline":0.52,"constant_mae_baseline":0.24733333333333335,"baseline_adjusted_skill":0.6097236456089223,"baseline_adjusted_score":0.6097236456089223}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/koniq-score / test · normalized_expected_score_mae. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.09404457839758337,"normalized_score_rmse":0.11366307028092944,"constant_score_baseline":0.578319964349225,"constant_mae_baseline":0.11212962028602311,"baseline_adjusted_skill":0.16128692706091352,"baseline_adjusted_score":0.16128692706091352}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/kuci-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.7666666666666667,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.25,"baseline_adjusted_skill":0.688888888888889,"baseline_adjusted_score":0.688888888888889}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/laya__ag_news / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.33,"baseline_adjusted_skill":0.8507462686567167,"baseline_adjusted_score":0.8507462686567167}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/laya__enron_spam / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.04950960698901004,"accuracy":0.94,"positive_recall":0.94,"specificity":0.94,"precision":0.94,"balanced_accuracy":0.94,"f1":0.94,"true_positive":47,"false_positive":3,"false_negative":3,"true_negative":47,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.8019615720439599,"baseline_adjusted_skill":0.8799999999999999,"baseline_adjusted_score":0.8799999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/laya__typed_decisions / test · target_mass_at_prediction. Population 100 cases / 148 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.5041588648648648,"uniform_accuracy_baseline":0.23445945945945948,"fixed_answer_accuracy_baseline":0.2623439054054054,"baseline_adjusted_skill":0.32781530747381343,"baseline_adjusted_score":0.32781530747381343}. Counts: {"cases": 100, "expected": 148, "succeeded": 148, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/laya__typed_decisions / test · binary_brier. Population 100 cases / 152 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.07886248415690521,"accuracy":0.6830701513157894,"positive_recall":0.9237028625423713,"specificity":0.45393438687134813,"precision":0.616966963963964,"balanced_accuracy":0.6888186247068597,"f1":0.7398004957361916,"true_positive":68.483333,"false_positive":42.516667,"false_negative":5.65667,"true_negative":35.34333,"positive_prevalence":0.4877631776315789,"majority_accuracy_baseline":0.512236822368421,"constant_brier_baseline":0.09451910039768556,"brier_skill":0.16564499847021108,"baseline_adjusted_skill":0.37763724941371946,"baseline_adjusted_score":0.37763724941371946}. Counts: {"cases": 100, "expected": 152, "succeeded": 152, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/laya__typed_decisions / test · normalized_expected_score_mae. Population 100 cases / 200 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.08470251935005085,"normalized_score_rmse":0.11723075071527232,"constant_score_baseline":0.5841668333333333,"constant_mae_baseline":0.21700552708333334,"baseline_adjusted_skill":0.6096757511732693,"baseline_adjusted_score":0.6096757511732693}. Counts: {"cases": 100, "expected": 200, "succeeded": 200, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/lexcomp / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.19069860000696853,"accuracy":0.7,"positive_recall":0.92,"specificity":0.48,"precision":0.6388888888888888,"balanced_accuracy":0.7,"f1":0.7540983606557377,"true_positive":46,"false_positive":26,"false_negative":4,"true_negative":24,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.23720559997212587,"baseline_adjusted_skill":0.3999999999999999,"baseline_adjusted_score":0.3999999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/logical_entailment / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.12142675891080376,"accuracy":0.81,"positive_recall":0.72,"specificity":0.9,"precision":0.8780487804878049,"balanced_accuracy":0.81,"f1":0.7912087912087912,"true_positive":36,"false_positive":5,"false_negative":14,"true_negative":45,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.514292964356785,"baseline_adjusted_skill":0.6200000000000001,"baseline_adjusted_score":0.6200000000000001}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/lonli / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.98,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.39,"baseline_adjusted_skill":0.9672131147540983,"baseline_adjusted_score":0.9672131147540983}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/massive / test · target_mass_at_prediction. Population 100 cases / 200 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.945,"uniform_accuracy_baseline":0.036111111111111115,"fixed_answer_accuracy_baseline":0.08833333333333333,"baseline_adjusted_skill":0.939670932358318,"baseline_adjusted_score":0.939670932358318}. Counts: {"cases": 100, "expected": 200, "succeeded": 200, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/mintaka-ja-boolean-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.20179378825133773,"accuracy":0.69,"positive_recall":0.7933333333333333,"specificity":0.5866666666666667,"precision":0.6574585635359116,"balanced_accuracy":0.69,"f1":0.7190332326283988,"true_positive":119,"false_positive":62,"false_negative":31,"true_negative":88,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.19282484699464908,"baseline_adjusted_skill":0.3799999999999999,"baseline_adjusted_score":0.3799999999999999}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/miqa / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.96,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.56,"baseline_adjusted_skill":0.909090909090909,"baseline_adjusted_score":0.909090909090909}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/mmlu-prox-ja-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.5066666666666667,"uniform_accuracy_baseline":0.11021428571428572,"fixed_answer_accuracy_baseline":0.12554761904761905,"baseline_adjusted_skill":0.4358373948321399,"baseline_adjusted_score":0.4358373948321399}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/mmmlu-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.6666666666666666,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.25,"baseline_adjusted_skill":0.5555555555555555,"baseline_adjusted_score":0.5555555555555555}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/mtop / test · target_mass_at_prediction. Population 100 cases / 200 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.875,"uniform_accuracy_baseline":0.049879324215607396,"fixed_answer_accuracy_baseline":0.12442477876106194,"baseline_adjusted_skill":0.8572367091166363,"baseline_adjusted_score":0.8572367091166363}. Counts: {"cases": 100, "expected": 200, "succeeded": 200, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/mvtec-ad-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.27173760511786677,"accuracy":0.5466666666666666,"positive_recall":0.375,"specificity":0.9880952380952381,"precision":0.9878048780487805,"balanced_accuracy":0.6815476190476191,"f1":0.5436241610738255,"true_positive":81,"false_positive":1,"false_negative":135,"true_negative":83,"positive_prevalence":0.72,"majority_accuracy_baseline":0.72,"constant_brier_baseline":0.2016,"brier_skill":-0.3479047872910057,"baseline_adjusted_skill":0.36309523809523814,"baseline_adjusted_score":0.36309523809523814}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/nights-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9433333333333334,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.5033333333333333,"baseline_adjusted_skill":0.8859060402684564,"baseline_adjusted_score":0.8859060402684564}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/nlsat / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.3947277295197691,"accuracy":0.51,"positive_recall":0.02,"specificity":1,"precision":1,"balanced_accuracy":0.51,"f1":0.0392156862745098,"true_positive":1,"false_positive":0,"false_negative":49,"true_negative":50,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":-0.5789109180790764,"baseline_adjusted_skill":0.020000000000000018,"baseline_adjusted_score":0.020000000000000018}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/nlvr2-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.05127424710983459,"accuracy":0.9366666666666666,"positive_recall":0.95,"specificity":0.9214285714285714,"precision":0.9325153374233128,"balanced_accuracy":0.9357142857142857,"f1":0.9411764705882353,"true_positive":152,"false_positive":11,"false_negative":8,"true_negative":129,"positive_prevalence":0.5333333333333333,"majority_accuracy_baseline":0.5333333333333333,"constant_brier_baseline":0.2488888888888889,"brier_skill":0.7939874000051289,"baseline_adjusted_skill":0.8714285714285714,"baseline_adjusted_score":0.8714285714285714}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__amount-extraction-control-v1 / test · target_mass_at_prediction. Population 36 cases / 36 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.8333333333333334,"uniform_accuracy_baseline":0.23888888888888887,"fixed_answer_accuracy_baseline":0.2916666666666667,"baseline_adjusted_skill":0.7647058823529413,"baseline_adjusted_score":0.7647058823529413}. Counts: {"cases": 36, "expected": 36, "succeeded": 36, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 36

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__amount-extraction-control-v1 / test · binary_brier. Population 64 cases / 64 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.049794436186587125,"accuracy":0.90625,"positive_recall":0.90625,"specificity":0.90625,"precision":0.90625,"balanced_accuracy":0.90625,"f1":0.90625,"true_positive":29,"false_positive":3,"false_negative":3,"true_negative":29,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.8008222552536515,"baseline_adjusted_skill":0.8125,"baseline_adjusted_score":0.8125}. Counts: {"cases": 64, "expected": 64, "succeeded": 64, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 64

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__browser-control-v1 / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":1,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.54,"baseline_adjusted_skill":1,"baseline_adjusted_score":1}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__citation-control-v1 / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.98,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.34,"baseline_adjusted_skill":0.9696969696969696,"baseline_adjusted_score":0.9696969696969696}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__context-retention-control-v1 / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.04220029719328272,"accuracy":0.93,"positive_recall":0.96,"specificity":0.9,"precision":0.9056603773584906,"balanced_accuracy":0.9299999999999999,"f1":0.9320388349514563,"true_positive":48,"false_positive":5,"false_negative":2,"true_negative":45,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.8311988112268691,"baseline_adjusted_skill":0.8599999999999999,"baseline_adjusted_score":0.8599999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__customer-control-v1 / test · target_mass_at_prediction. Population 17 cases / 17 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.8823529411764706,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.3235294117647059,"baseline_adjusted_skill":0.8260869565217391,"baseline_adjusted_score":0.8260869565217391}. Counts: {"cases": 17, "expected": 17, "succeeded": 17, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 17

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__customer-control-v1 / test · binary_brier. Population 33 cases / 33 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.003450211022679325,"accuracy":1,"positive_recall":1,"specificity":1,"precision":1,"balanced_accuracy":1,"f1":1,"true_positive":16,"false_positive":0,"false_negative":0,"true_negative":17,"positive_prevalence":0.48484848484848486,"majority_accuracy_baseline":0.5151515151515151,"constant_brier_baseline":0.2497704315886134,"brier_skill":0.9861864713099346,"baseline_adjusted_skill":1,"baseline_adjusted_score":1}. Counts: {"cases": 33, "expected": 33, "succeeded": 33, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 33

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__customer-control-v1 / test · normalized_expected_score_mae. Population 50 cases / 50 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.15771777232869702,"normalized_score_rmse":0.3153981072922279,"constant_score_baseline":0.25,"constant_mae_baseline":0.33,"baseline_adjusted_skill":0.522067356579706,"baseline_adjusted_score":0.522067356579706}. Counts: {"cases": 50, "expected": 50, "succeeded": 50, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 50

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__drone-control-v1 / test · target_mass_at_prediction. Population 32 cases / 32 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.375,"uniform_accuracy_baseline":0.178125,"fixed_answer_accuracy_baseline":0.2875,"baseline_adjusted_skill":0.12280701754385967,"baseline_adjusted_score":0.12280701754385967}. Counts: {"cases": 32, "expected": 32, "succeeded": 32, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 32

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__drone-control-v1 / test · binary_brier. Population 34 cases / 34 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.0004994181867421103,"accuracy":1,"positive_recall":1,"specificity":1,"precision":1,"balanced_accuracy":1,"f1":1,"true_positive":17,"false_positive":0,"false_negative":0,"true_negative":17,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.9980023272530315,"baseline_adjusted_skill":1,"baseline_adjusted_score":1}. Counts: {"cases": 34, "expected": 34, "succeeded": 34, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 34

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__drone-control-v1 / test · normalized_expected_score_mae. Population 34 cases / 34 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.19164828174063073,"normalized_score_rmse":0.23914210983182027,"constant_score_baseline":0.5,"constant_mae_baseline":0.14705882352941177,"baseline_adjusted_skill":-0.3032083158362888,"baseline_adjusted_score":0}. Counts: {"cases": 34, "expected": 34, "succeeded": 34, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 34

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__email-selection-control-v1 / test · target_mass_at_prediction. Population 48 cases / 48 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.8958333333333334,"uniform_accuracy_baseline":0.13194444444444445,"fixed_answer_accuracy_baseline":0.6006944444444444,"baseline_adjusted_skill":0.7391304347826088,"baseline_adjusted_score":0.7391304347826088}. Counts: {"cases": 48, "expected": 48, "succeeded": 48, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 48

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__email-selection-control-v1 / test · binary_brier. Population 52 cases / 52 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.000262202322402121,"accuracy":1,"positive_recall":1,"specificity":1,"precision":1,"balanced_accuracy":1,"f1":1,"true_positive":26,"false_positive":0,"false_negative":0,"true_negative":26,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.9989511907103915,"baseline_adjusted_skill":1,"baseline_adjusted_score":1}. Counts: {"cases": 52, "expected": 52, "succeeded": 52, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 52

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__entity-alignment-control-v1 / test · binary_brier. Population 75 cases / 75 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.06350741892254216,"accuracy":0.9066666666666666,"positive_recall":1,"specificity":0.8055555555555556,"precision":0.8478260869565217,"balanced_accuracy":0.9027777777777778,"f1":0.9176470588235294,"true_positive":39,"false_positive":7,"false_negative":0,"true_negative":29,"positive_prevalence":0.52,"majority_accuracy_baseline":0.52,"constant_brier_baseline":0.24960000000000004,"brier_skill":0.7455632254705844,"baseline_adjusted_skill":0.8055555555555556,"baseline_adjusted_score":0.8055555555555556}. Counts: {"cases": 75, "expected": 75, "succeeded": 75, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 75

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__entity-alignment-control-v1 / test · normalized_expected_score_mae. Population 25 cases / 25 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.3174087399248463,"normalized_score_rmse":0.43573916529418416,"constant_score_baseline":0.5,"constant_mae_baseline":0.28,"baseline_adjusted_skill":-0.13360264258873666,"baseline_adjusted_score":0}. Counts: {"cases": 25, "expected": 25, "succeeded": 25, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 25

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__ir-control-v1 / test · target_mass_at_prediction. Population 17 cases / 17 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.8823529411764706,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.2647058823529412,"baseline_adjusted_skill":0.84,"baseline_adjusted_score":0.84}. Counts: {"cases": 17, "expected": 17, "succeeded": 17, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 17

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__ir-control-v1 / test · binary_brier. Population 28 cases / 28 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.08348644180435418,"accuracy":0.8928571428571429,"positive_recall":1,"specificity":0.7857142857142857,"precision":0.8235294117647058,"balanced_accuracy":0.8928571428571428,"f1":0.9032258064516129,"true_positive":14,"false_positive":3,"false_negative":0,"true_negative":11,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.6660542327825834,"baseline_adjusted_skill":0.7857142857142856,"baseline_adjusted_score":0.7857142857142856}. Counts: {"cases": 28, "expected": 28, "succeeded": 28, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 28

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__ir-control-v1 / test · normalized_expected_score_mae. Population 55 cases / 55 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.04652219080762821,"normalized_score_rmse":0.07775717607319904,"constant_score_baseline":0.3333333333333333,"constant_mae_baseline":0.1696969696969697,"baseline_adjusted_skill":0.7258513755979052,"baseline_adjusted_score":0.7258513755979052}. Counts: {"cases": 55, "expected": 55, "succeeded": 55, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 55

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__mailroom-control-v1 / test · target_mass_at_prediction. Population 15 cases / 15 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9333333333333333,"uniform_accuracy_baseline":0.15555555555555553,"fixed_answer_accuracy_baseline":0.2,"baseline_adjusted_skill":0.9166666666666667,"baseline_adjusted_score":0.9166666666666667}. Counts: {"cases": 15, "expected": 15, "succeeded": 15, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 15

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__mailroom-control-v1 / test · binary_brier. Population 85 cases / 85 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.030982124937071292,"accuracy":0.9647058823529412,"positive_recall":1,"specificity":0.925,"precision":0.9375,"balanced_accuracy":0.9625,"f1":0.967741935483871,"true_positive":45,"false_positive":3,"false_negative":0,"true_negative":37,"positive_prevalence":0.5294117647058824,"majority_accuracy_baseline":0.5294117647058824,"constant_brier_baseline":0.2491349480968858,"brier_skill":0.8756411929609221,"baseline_adjusted_skill":0.925,"baseline_adjusted_score":0.925}. Counts: {"cases": 85, "expected": 85, "succeeded": 85, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 85

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__painting-geometry-v1 / test · target_mass_at_prediction. Population 25 cases / 25 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":1,"uniform_accuracy_baseline":0.07805555555555556,"fixed_answer_accuracy_baseline":0.24,"baseline_adjusted_skill":1,"baseline_adjusted_score":1}. Counts: {"cases": 25, "expected": 25, "succeeded": 25, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 25

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__painting-geometry-v1 / test · binary_brier. Population 50 cases / 50 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.2788454838852676,"accuracy":0.68,"positive_recall":0.5714285714285714,"specificity":0.9333333333333333,"precision":0.9523809523809523,"balanced_accuracy":0.7523809523809524,"f1":0.7142857142857143,"true_positive":20,"false_positive":1,"false_negative":15,"true_negative":14,"positive_prevalence":0.7,"majority_accuracy_baseline":0.7,"constant_brier_baseline":0.21,"brier_skill":-0.32783563754889333,"baseline_adjusted_skill":0.5047619047619047,"baseline_adjusted_score":0.5047619047619047}. Counts: {"cases": 50, "expected": 50, "succeeded": 50, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 50

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__painting-geometry-v1 / test · normalized_expected_score_mae. Population 25 cases / 25 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.09498493354873849,"normalized_score_rmse":0.2358662847266609,"constant_score_baseline":0.5,"constant_mae_baseline":0.28,"baseline_adjusted_skill":0.6607680944687911,"baseline_adjusted_score":0.6607680944687911}. Counts: {"cases": 25, "expected": 25, "succeeded": 25, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 25

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__phone-extraction-control-v1 / test · target_mass_at_prediction. Population 79 cases / 79 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9493670886075949,"uniform_accuracy_baseline":0.24240506329113923,"fixed_answer_accuracy_baseline":0.4177215189873418,"baseline_adjusted_skill":0.9130434782608695,"baseline_adjusted_score":0.9130434782608695}. Counts: {"cases": 79, "expected": 79, "succeeded": 79, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 79

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__phone-extraction-control-v1 / test · binary_brier. Population 21 cases / 21 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.0005324347530619907,"accuracy":1,"positive_recall":1,"specificity":1,"precision":1,"balanced_accuracy":1,"f1":1,"true_positive":11,"false_positive":0,"false_negative":0,"true_negative":10,"positive_prevalence":0.5238095238095238,"majority_accuracy_baseline":0.5238095238095238,"constant_brier_baseline":0.2494331065759637,"brier_skill":0.9978654206718152,"baseline_adjusted_skill":1,"baseline_adjusted_score":1}. Counts: {"cases": 21, "expected": 21, "succeeded": 21, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 21

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__reasoning-control-v1 / test · target_mass_at_prediction. Population 33 cases / 33 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9393939393939394,"uniform_accuracy_baseline":0.32373737373737377,"fixed_answer_accuracy_baseline":0.3939393939393939,"baseline_adjusted_skill":0.9000000000000001,"baseline_adjusted_score":0.9000000000000001}. Counts: {"cases": 33, "expected": 33, "succeeded": 33, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 33

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__reasoning-control-v1 / test · binary_brier. Population 33 cases / 33 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.19977791852915625,"accuracy":0.7878787878787878,"positive_recall":0.6470588235294118,"specificity":0.9375,"precision":0.9166666666666666,"balanced_accuracy":0.7922794117647058,"f1":0.7586206896551724,"true_positive":11,"false_positive":1,"false_negative":6,"true_negative":15,"positive_prevalence":0.5151515151515151,"majority_accuracy_baseline":0.5151515151515151,"constant_brier_baseline":0.2497704315886134,"brier_skill":0.20015384824172366,"baseline_adjusted_skill":0.5845588235294117,"baseline_adjusted_score":0.5845588235294117}. Counts: {"cases": 33, "expected": 33, "succeeded": 33, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 33

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__reasoning-control-v1 / test · normalized_expected_score_mae. Population 34 cases / 34 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.04477235975075581,"normalized_score_rmse":0.16472630070458763,"constant_score_baseline":0.7083333333333333,"constant_mae_baseline":0.3431372549019608,"baseline_adjusted_skill":0.8695205515835116,"baseline_adjusted_score":0.8695205515835116}. Counts: {"cases": 34, "expected": 34, "succeeded": 34, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 34

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__silent-failure-control-v1 / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.04469626898202165,"accuracy":0.95,"positive_recall":0.98,"specificity":0.92,"precision":0.9245283018867925,"balanced_accuracy":0.95,"f1":0.9514563106796117,"true_positive":49,"false_positive":4,"false_negative":1,"true_negative":46,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.8212149240719134,"baseline_adjusted_skill":0.8999999999999999,"baseline_adjusted_score":0.8999999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__snake-v1 / test · target_mass_at_prediction. Population 25 cases / 25 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.5733333333333334,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.35333333333333333,"baseline_adjusted_skill":0.3402061855670103,"baseline_adjusted_score":0.3402061855670103}. Counts: {"cases": 25, "expected": 25, "succeeded": 25, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 25

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__snake-v1 / test · binary_brier. Population 75 cases / 75 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.22367027522752536,"accuracy":0.6,"positive_recall":0.35135135135135137,"specificity":0.8421052631578947,"precision":0.6842105263157895,"balanced_accuracy":0.596728307254623,"f1":0.4642857142857143,"true_positive":13,"false_positive":6,"false_negative":24,"true_negative":32,"positive_prevalence":0.49333333333333335,"majority_accuracy_baseline":0.5066666666666666,"constant_brier_baseline":0.24995555555555554,"brier_skill":0.10515981639059013,"baseline_adjusted_skill":0.193456614509246,"baseline_adjusted_score":0.193456614509246}. Counts: {"cases": 75, "expected": 75, "succeeded": 75, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 75

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__sponsor-segment-control-v1 / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.87,"uniform_accuracy_baseline":0.14285714285714285,"fixed_answer_accuracy_baseline":0.22,"baseline_adjusted_skill":0.8333333333333334,"baseline_adjusted_score":0.8333333333333334}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__tic_tac_toe-v1 / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.355,"uniform_accuracy_baseline":0.32416666666666666,"fixed_answer_accuracy_baseline":0.34,"baseline_adjusted_skill":0.022727272727272665,"baseline_adjusted_score":0.022727272727272665}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__vizdoom-basic-v1 / test · binary_brier. Population 33 cases / 33 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.12174477051626553,"accuracy":0.8484848484848485,"positive_recall":1,"specificity":0.7058823529411765,"precision":0.7619047619047619,"balanced_accuracy":0.8529411764705883,"f1":0.8648648648648649,"true_positive":16,"false_positive":5,"false_negative":0,"true_negative":12,"positive_prevalence":0.48484848484848486,"majority_accuracy_baseline":0.5151515151515151,"constant_brier_baseline":0.2497704315886134,"brier_skill":0.5125733268668633,"baseline_adjusted_skill":0.7058823529411766,"baseline_adjusted_score":0.7058823529411766}. Counts: {"cases": 33, "expected": 33, "succeeded": 33, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 33

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__vizdoom-basic-v1 / test · normalized_expected_score_mae. Population 33 cases / 33 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.021631874561344414,"normalized_score_rmse":0.03977451940388981,"constant_score_baseline":0.5,"constant_mae_baseline":0.2196969696969697,"baseline_adjusted_skill":0.9015376744104323,"baseline_adjusted_score":0.9015376744104323}. Counts: {"cases": 33, "expected": 33, "succeeded": 33, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 33

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__workflow-controls-v1__agent_trace_observability / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.16040710722609408,"accuracy":0.78,"positive_recall":0.6818181818181818,"specificity":0.8076923076923077,"precision":0.5,"balanced_accuracy":0.7447552447552448,"f1":0.5769230769230769,"true_positive":15,"false_positive":15,"false_negative":7,"true_negative":63,"positive_prevalence":0.22,"majority_accuracy_baseline":0.78,"constant_brier_baseline":0.1716,"brier_skill":0.06522664786658461,"baseline_adjusted_skill":0.4895104895104896,"baseline_adjusted_score":0.4895104895104896}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__workflow-controls-v1__customer_service / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.11201783924487645,"accuracy":0.83,"positive_recall":0.5675675675675675,"specificity":0.9841269841269841,"precision":0.9545454545454546,"balanced_accuracy":0.7758472758472759,"f1":0.711864406779661,"true_positive":21,"false_positive":1,"false_negative":16,"true_negative":62,"positive_prevalence":0.37,"majority_accuracy_baseline":0.63,"constant_brier_baseline":0.23310000000000003,"brier_skill":0.5194429890824692,"baseline_adjusted_skill":0.5516945516945517,"baseline_adjusted_score":0.5516945516945517}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__workflow-controls-v1__invoice_processing / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.060470134208703195,"accuracy":0.93,"positive_recall":0.8205128205128205,"specificity":1,"precision":1,"balanced_accuracy":0.9102564102564102,"f1":0.9014084507042254,"true_positive":32,"false_positive":0,"false_negative":7,"true_negative":61,"positive_prevalence":0.39,"majority_accuracy_baseline":0.61,"constant_brier_baseline":0.2379,"brier_skill":0.7458170062685868,"baseline_adjusted_skill":0.8205128205128205,"baseline_adjusted_score":0.8205128205128205}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/open_jev__workflow-controls-v1__security_incidents / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.07142106013290345,"accuracy":0.9,"positive_recall":0.9032258064516129,"specificity":0.8985507246376812,"precision":0.8,"balanced_accuracy":0.9008882655446471,"f1":0.8484848484848485,"true_positive":28,"false_positive":7,"false_negative":3,"true_negative":62,"positive_prevalence":0.31,"majority_accuracy_baseline":0.69,"constant_brier_baseline":0.21389999999999998,"brier_skill":0.6661007006409376,"baseline_adjusted_skill":0.8017765310892941,"baseline_adjusted_score":0.8017765310892941}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/openbookqa / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.96,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.28,"baseline_adjusted_skill":0.9444444444444444,"baseline_adjusted_score":0.9444444444444444}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/patent_similarity / test · normalized_expected_score_mae. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.12893838944619643,"normalized_score_rmse":0.1606639917529075,"constant_score_baseline":0.25,"constant_mae_baseline":0.2325,"baseline_adjusted_skill":0.4454262819518433,"baseline_adjusted_score":0.4454262819518433}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/paws / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.045013098931200195,"accuracy":0.95,"positive_recall":0.98,"specificity":0.92,"precision":0.9245283018867925,"balanced_accuracy":0.95,"f1":0.9514563106796117,"true_positive":49,"false_positive":4,"false_negative":1,"true_negative":46,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.8199476042751992,"baseline_adjusted_skill":0.8999999999999999,"baseline_adjusted_score":0.8999999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/plane / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.3156101705376605,"accuracy":0.6,"positive_recall":0.84,"specificity":0.36,"precision":0.5675675675675675,"balanced_accuracy":0.6,"f1":0.6774193548387096,"true_positive":42,"false_positive":32,"false_negative":8,"true_negative":18,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":-0.262440682150642,"baseline_adjusted_skill":0.19999999999999996,"baseline_adjusted_score":0.19999999999999996}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/poem_sentiment / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.87,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.65,"baseline_adjusted_skill":0.6285714285714286,"baseline_adjusted_score":0.6285714285714286}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/pope-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.08642839398401847,"accuracy":0.8866666666666667,"positive_recall":0.7933333333333333,"specificity":0.98,"precision":0.9754098360655737,"balanced_accuracy":0.8866666666666667,"f1":0.875,"true_positive":119,"false_positive":3,"false_negative":31,"true_negative":147,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.6542864240639261,"baseline_adjusted_skill":0.7733333333333334,"baseline_adjusted_score":0.7733333333333334}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/qasper / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.11010027144049665,"accuracy":0.84,"positive_recall":0.94,"specificity":0.74,"precision":0.7833333333333333,"balanced_accuracy":0.84,"f1":0.8545454545454545,"true_positive":47,"false_positive":13,"false_negative":3,"true_negative":37,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.5595989142380133,"baseline_adjusted_skill":0.6799999999999999,"baseline_adjusted_score":0.6799999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/quartz / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.94,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.58,"baseline_adjusted_skill":0.857142857142857,"baseline_adjusted_score":0.857142857142857}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/robust_lr / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.91,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.4,"baseline_adjusted_skill":0.8500000000000001,"baseline_adjusted_score":0.8500000000000001}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/ruletaker / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.08176071967641355,"accuracy":0.87,"positive_recall":0.76,"specificity":0.98,"precision":0.9743589743589743,"balanced_accuracy":0.87,"f1":0.8539325842696629,"true_positive":38,"false_positive":1,"false_negative":12,"true_negative":49,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.6729571212943458,"baseline_adjusted_skill":0.74,"baseline_adjusted_score":0.74}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/rvlcdip-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.7266666666666667,"uniform_accuracy_baseline":0.0625,"fixed_answer_accuracy_baseline":0.10666666666666667,"baseline_adjusted_skill":0.6940298507462687,"baseline_adjusted_score":0.6940298507462687}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/s1mb-generalization-contextual-choice / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":1,"uniform_accuracy_baseline":0.20851190476190473,"fixed_answer_accuracy_baseline":0.22333333333333333,"baseline_adjusted_skill":1,"baseline_adjusted_score":1}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/s1mb-generalization-contextual-noul / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.021533383920991276,"accuracy":0.97,"positive_recall":0.98,"specificity":0.96,"precision":0.9607843137254902,"balanced_accuracy":0.97,"f1":0.9702970297029703,"true_positive":49,"false_positive":2,"false_negative":1,"true_negative":48,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.9138664643160349,"baseline_adjusted_skill":0.94,"baseline_adjusted_score":0.94}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/s1mb-generalization-contextual-score / test · normalized_expected_score_mae. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.033839400387713776,"normalized_score_rmse":0.08117088261428208,"constant_score_baseline":0.5,"constant_mae_baseline":0.3,"baseline_adjusted_skill":0.8872019987076207,"baseline_adjusted_score":0.8872019987076207}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/s1mb-generalization-diverse-choice / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":1,"uniform_accuracy_baseline":0.22851190476190475,"fixed_answer_accuracy_baseline":0.24333333333333332,"baseline_adjusted_skill":1,"baseline_adjusted_score":1}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/s1mb-generalization-diverse-noul / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.002472000136244679,"accuracy":1,"positive_recall":1,"specificity":1,"precision":1,"balanced_accuracy":1,"f1":1,"true_positive":50,"false_positive":0,"false_negative":0,"true_negative":50,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.9901119994550213,"baseline_adjusted_skill":1,"baseline_adjusted_score":1}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/s1mb-generalization-diverse-score / test · normalized_expected_score_mae. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.056125286209200057,"normalized_score_rmse":0.1052706014384171,"constant_score_baseline":0.4222222222222222,"constant_mae_baseline":0.3042222222222222,"baseline_adjusted_skill":0.8155122074934987,"baseline_adjusted_score":0.8155122074934987}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/scicite / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.79,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.38,"baseline_adjusted_skill":0.6612903225806452,"baseline_adjusted_score":0.6612903225806452}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/scienceqa-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.9833333333333333,"uniform_accuracy_baseline":0.3467777777777778,"fixed_answer_accuracy_baseline":0.36,"baseline_adjusted_skill":0.9739583333333333,"baseline_adjusted_score":0.9739583333333333}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/scitail / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.04700728182027222,"accuracy":0.94,"positive_recall":1,"specificity":0.88,"precision":0.8928571428571429,"balanced_accuracy":0.94,"f1":0.9433962264150944,"true_positive":50,"false_positive":6,"false_negative":0,"true_negative":44,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.8119708727189111,"baseline_adjusted_skill":0.8799999999999999,"baseline_adjusted_score":0.8799999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/scone / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.96,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.5,"baseline_adjusted_skill":0.9199999999999999,"baseline_adjusted_score":0.9199999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/sdoh_nli / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.012589625826572266,"accuracy":1,"positive_recall":1,"specificity":1,"precision":1,"balanced_accuracy":1,"f1":1,"true_positive":50,"false_positive":0,"false_negative":0,"true_negative":50,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.9496414966937109,"baseline_adjusted_skill":1,"baseline_adjusted_score":1}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/sgd / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.83,"uniform_accuracy_baseline":0.3533333333333333,"fixed_answer_accuracy_baseline":0.41,"baseline_adjusted_skill":0.7118644067796609,"baseline_adjusted_score":0.7118644067796609}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/snli / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.98,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.38,"baseline_adjusted_skill":0.9677419354838709,"baseline_adjusted_score":0.9677419354838709}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/sobaco-culture-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.5066666666666667,"uniform_accuracy_baseline":0.3333333333333333,"fixed_answer_accuracy_baseline":0.3333333333333333,"baseline_adjusted_skill":0.26000000000000006,"baseline_adjusted_score":0.26000000000000006}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/spartqa / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.91,"uniform_accuracy_baseline":0.25,"fixed_answer_accuracy_baseline":0.38,"baseline_adjusted_skill":0.8548387096774194,"baseline_adjusted_score":0.8548387096774194}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/stepgame / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.64,"uniform_accuracy_baseline":0.1111111111111111,"fixed_answer_accuracy_baseline":0.16,"baseline_adjusted_skill":0.5714285714285714,"baseline_adjusted_score":0.5714285714285714}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/sugarcrepe-choice / test · target_mass_at_prediction. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.97,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.58,"baseline_adjusted_skill":0.9285714285714285,"baseline_adjusted_score":0.9285714285714285}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoArguAna / test · binary_brier. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.12448231071004054,"accuracy":0.6878749999999999,"positive_recall":0.6064601186552406,"specificity":0.7069258059540336,"precision":0.3262411347517731,"balanced_accuracy":0.6566929623046371,"f1":0.42425639843209595,"true_positive":46,"false_positive":95,"false_negative":29.85,"true_negative":229.15,"positive_prevalence":0.189625,"majority_accuracy_baseline":0.8103750000000001,"constant_brier_baseline":0.07612360937500001,"brier_skill":-0.6352654811310374,"baseline_adjusted_skill":0.3133859246092743,"baseline_adjusted_score":0.3133859246092743}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoArguAna / test · normalized_expected_score_mae. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.18922042565916264,"normalized_score_rmse":0.2582164587672029,"constant_score_baseline":0,"constant_mae_baseline":0.21375,"baseline_adjusted_skill":0.11475824253023326,"baseline_adjusted_score":0.11475824253023326}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoDBPedia / test · binary_brier. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.061945751977668516,"accuracy":0.7923749999999999,"positive_recall":0.5091420534458508,"specificity":0.895346855292313,"precision":0.6388235294117647,"balanced_accuracy":0.7022444543690819,"f1":0.5666579702582833,"true_positive":54.3,"false_positive":30.7,"false_negative":52.35,"true_negative":262.65,"positive_prevalence":0.266625,"majority_accuracy_baseline":0.733375,"constant_brier_baseline":0.115654859375,"brier_skill":0.46439127320353013,"baseline_adjusted_skill":0.4044889087381638,"baseline_adjusted_score":0.4044889087381638}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoDBPedia / test · normalized_expected_score_mae. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.1654319397701889,"normalized_score_rmse":0.2180143906243523,"constant_score_baseline":0.25,"constant_mae_baseline":0.26875,"baseline_adjusted_skill":0.38443929387836684,"baseline_adjusted_score":0.38443929387836684}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoFiQA2018 / test · binary_brier. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.05341428303200631,"accuracy":0.784125,"positive_recall":0.69082774049217,"specificity":0.820294882914137,"precision":0.5984496124031008,"balanced_accuracy":0.7555613117031534,"f1":0.6413291796469367,"true_positive":77.2,"false_positive":51.8,"false_negative":34.55,"true_negative":236.45,"positive_prevalence":0.279375,"majority_accuracy_baseline":0.7206250000000001,"constant_brier_baseline":0.11315585937500001,"brier_skill":0.5279583105370561,"baseline_adjusted_skill":0.5111226234063069,"baseline_adjusted_score":0.5111226234063069}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoFiQA2018 / test · normalized_expected_score_mae. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.16272157744957746,"normalized_score_rmse":0.20781401477570557,"constant_score_baseline":0.25,"constant_mae_baseline":0.264375,"baseline_adjusted_skill":0.3845046715855227,"baseline_adjusted_score":0.3845046715855227}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoHotpotQA / test · binary_brier. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.04298452620851991,"accuracy":0.8798750000000001,"positive_recall":0.6011267605633803,"specificity":0.9593574297188756,"precision":0.8083333333333333,"balanced_accuracy":0.780242095141128,"f1":0.6894991922455573,"true_positive":53.35,"false_positive":12.65,"false_negative":35.4,"true_negative":298.6,"positive_prevalence":0.221875,"majority_accuracy_baseline":0.778125,"constant_brier_baseline":0.12674023437499998,"brier_skill":0.6608454574785071,"baseline_adjusted_skill":0.5604841902822559,"baseline_adjusted_score":0.5604841902822559}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoHotpotQA / test · normalized_expected_score_mae. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.14299740913369405,"normalized_score_rmse":0.20287020622177454,"constant_score_baseline":0,"constant_mae_baseline":0.225625,"baseline_adjusted_skill":0.36621646921354434,"baseline_adjusted_score":0.36621646921354434}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoMSMARCO / test · binary_brier. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.03617131961469424,"accuracy":0.83325,"positive_recall":0.751890359168242,"specificity":0.8625084976206663,"precision":0.6629166666666666,"balanced_accuracy":0.8071994283944541,"f1":0.7046058458813108,"true_positive":79.55,"false_positive":40.45,"false_negative":26.25,"true_negative":253.75,"positive_prevalence":0.2645,"majority_accuracy_baseline":0.7355,"constant_brier_baseline":0.11362725,"brier_skill":0.6816668570726279,"baseline_adjusted_skill":0.6143988567889083,"baseline_adjusted_score":0.6143988567889083}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoMSMARCO / test · normalized_expected_score_mae. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.14009826392603306,"normalized_score_rmse":0.18381275389860638,"constant_score_baseline":0.25,"constant_mae_baseline":0.25125,"baseline_adjusted_skill":0.44239496944862455,"baseline_adjusted_score":0.44239496944862455}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoNFCorpus / test · binary_brier. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.05790152237537661,"accuracy":0.82325,"positive_recall":0.42362924281984343,"specificity":0.9179035250463822,"precision":0.55,"balanced_accuracy":0.6707663839331128,"f1":0.478613569321534,"true_positive":32.45,"false_positive":26.55,"false_negative":44.15,"true_negative":296.85,"positive_prevalence":0.19149999999999998,"majority_accuracy_baseline":0.8085,"constant_brier_baseline":0.09190274999999999,"brier_skill":0.36996964317850534,"baseline_adjusted_skill":0.34153276786622566,"baseline_adjusted_score":0.34153276786622566}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoNFCorpus / test · normalized_expected_score_mae. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.14063005315817617,"normalized_score_rmse":0.2148853347738841,"constant_score_baseline":0,"constant_mae_baseline":0.206875,"baseline_adjusted_skill":0.320217265700659,"baseline_adjusted_score":0.320217265700659}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoNQ / test · binary_brier. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.047026573458263325,"accuracy":0.8789999999999999,"positive_recall":0.7547425474254742,"specificity":0.9071122011036173,"precision":0.6476744186046512,"balanced_accuracy":0.8309273742645458,"f1":0.6971214017521902,"true_positive":55.7,"false_positive":30.3,"false_negative":18.1,"true_negative":295.9,"positive_prevalence":0.1845,"majority_accuracy_baseline":0.8155,"constant_brier_baseline":0.10922225,"brier_skill":0.5694414511854194,"baseline_adjusted_skill":0.6618547485290915,"baseline_adjusted_score":0.6618547485290915}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoNQ / test · normalized_expected_score_mae. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.15598077216439235,"normalized_score_rmse":0.22361710542756666,"constant_score_baseline":0,"constant_mae_baseline":0.189375,"baseline_adjusted_skill":0.17633915688769708,"baseline_adjusted_score":0.17633915688769708}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoSCIDOCS / test · binary_brier. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.06552918560769512,"accuracy":0.8290000000000001,"positive_recall":0.17898089171974524,"specificity":0.9877138413685848,"precision":0.7805555555555556,"balanced_accuracy":0.583347366544165,"f1":0.2911917098445596,"true_positive":14.05,"false_positive":3.95,"false_negative":64.45,"true_negative":317.55,"positive_prevalence":0.19625,"majority_accuracy_baseline":0.80375,"constant_brier_baseline":0.09878593749999999,"brier_skill":0.33665471760395926,"baseline_adjusted_skill":0.16669473308832994,"baseline_adjusted_score":0.16669473308832994}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoSCIDOCS / test · normalized_expected_score_mae. Population 400 cases / 400 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.1676091485552786,"normalized_score_rmse":0.2628937894715782,"constant_score_baseline":0,"constant_mae_baseline":0.21125,"baseline_adjusted_skill":0.20658391216436167,"baseline_adjusted_score":0.20658391216436167}. Counts: {"cases": 400, "expected": 400, "succeeded": 400, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 400

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoTouche2020 / test · binary_brier. Population 392 cases / 392 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.06408913249705565,"accuracy":0.7122448979591838,"positive_recall":0.8683274021352314,"specificity":0.594449418084154,"precision":0.6177215189873418,"balanced_accuracy":0.7313884101096927,"f1":0.7218934911242605,"true_positive":146.4,"false_positive":90.6,"false_negative":22.2,"true_negative":132.8,"positive_prevalence":0.4301020408163265,"majority_accuracy_baseline":0.5698979591836735,"constant_brier_baseline":0.12359641815910039,"brier_skill":0.4814644837477696,"baseline_adjusted_skill":0.4627768202193854,"baseline_adjusted_score":0.4627768202193854}. Counts: {"cases": 392, "expected": 392, "succeeded": 392, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 392

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanobeir__NanoTouche2020 / test · normalized_expected_score_mae. Population 392 cases / 392 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.15221271845691123,"normalized_score_rmse":0.19688734030707028,"constant_score_baseline":0.5,"constant_mae_baseline":0.2755102040816326,"baseline_adjusted_skill":0.4475242070823221,"baseline_adjusted_score":0.4475242070823221}. Counts: {"cases": 392, "expected": 392, "succeeded": 392, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 392

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanocoir / test · binary_brier. Population 800 cases / 800 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.0509067858900383,"accuracy":0.8828750000000001,"positive_recall":0.7254130605822188,"specificity":0.9126170307623719,"precision":0.6105960264900663,"balanced_accuracy":0.8190150456722953,"f1":0.6630708378281195,"true_positive":92.2,"false_positive":58.8,"false_negative":34.9,"true_negative":614.1,"positive_prevalence":0.158875,"majority_accuracy_baseline":0.841125,"constant_brier_baseline":0.09877748437499999,"brier_skill":0.48463168289672987,"baseline_adjusted_skill":0.6380300913445907,"baseline_adjusted_score":0.6380300913445907}. Counts: {"cases": 800, "expected": 800, "succeeded": 800, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 800

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/synthetic_relevance__nanocoir / test · normalized_expected_score_mae. Population 800 cases / 800 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.14889705828884967,"normalized_score_rmse":0.21858522081117582,"constant_score_baseline":0,"constant_mae_baseline":0.165,"baseline_adjusted_skill":0.09759358612818392,"baseline_adjusted_score":0.09759358612818392}. Counts: {"cases": 800, "expected": 800, "succeeded": 800, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 800

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/tallyqa-score / test · normalized_expected_score_mae. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"normalized_expected_score_mae":0.033226027769752725,"normalized_score_rmse":0.06728901532977062,"constant_score_baseline":0.1111111111111111,"constant_mae_baseline":0.10629629629629629,"baseline_adjusted_skill":0.6874206446748001,"baseline_adjusted_score":0.6874206446748001}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/temporal_nli / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.95,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.38,"baseline_adjusted_skill":0.9193548387096774,"baseline_adjusted_score":0.9193548387096774}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/tracie / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.17404021111717444,"accuracy":0.75,"positive_recall":0.88,"specificity":0.62,"precision":0.6984126984126984,"balanced_accuracy":0.75,"f1":0.7787610619469026,"true_positive":44,"false_positive":19,"false_negative":6,"true_negative":31,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.30383915553130225,"baseline_adjusted_skill":0.5,"baseline_adjusted_score":0.5}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/ud_ewt / test · target_mass_at_prediction. Population 100 cases / 1509 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.2829688535453943,"uniform_accuracy_baseline":0.058823529411764705,"fixed_answer_accuracy_baseline":0.14910536779324055,"baseline_adjusted_skill":0.15732087227414332,"baseline_adjusted_score":0.15732087227414332}. Counts: {"cases": 100, "expected": 1509, "succeeded": 1509, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/visa-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.30054188660935616,"accuracy":0.53,"positive_recall":0.1896551724137931,"specificity":1,"precision":1,"balanced_accuracy":0.5948275862068966,"f1":0.3188405797101449,"true_positive":33,"false_positive":0,"false_negative":141,"true_negative":126,"positive_prevalence":0.58,"majority_accuracy_baseline":0.58,"constant_brier_baseline":0.24359999999999998,"brier_skill":-0.23375158706632249,"baseline_adjusted_skill":0.18965517241379315,"baseline_adjusted_score":0.18965517241379315}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/vsr-noul / test · binary_brier. Population 300 cases / 300 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.11746211391164675,"accuracy":0.8133333333333334,"positive_recall":0.8789808917197452,"specificity":0.7412587412587412,"precision":0.7885714285714286,"balanced_accuracy":0.8101198164892432,"f1":0.8313253012048193,"true_positive":138,"false_positive":37,"false_negative":19,"true_negative":106,"positive_prevalence":0.5233333333333333,"majority_accuracy_baseline":0.5233333333333333,"constant_brier_baseline":0.24945555555555557,"brier_skill":0.5291260856065116,"baseline_adjusted_skill":0.6202396329784865,"baseline_adjusted_score":0.6202396329784865}. Counts: {"cases": 300, "expected": 300, "succeeded": 300, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 300

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/winobias / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.93,"uniform_accuracy_baseline":0.5,"fixed_answer_accuracy_baseline":0.54,"baseline_adjusted_skill":0.8478260869565218,"baseline_adjusted_score":0.8478260869565218}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/winowhy / test · binary_brier. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. All published diagnostics: {"binary_brier":0.13778387658840222,"accuracy":0.83,"positive_recall":0.94,"specificity":0.72,"precision":0.7704918032786885,"balanced_accuracy":0.83,"f1":0.8468468468468469,"true_positive":47,"false_positive":14,"false_negative":3,"true_negative":36,"positive_prevalence":0.5,"majority_accuracy_baseline":0.5,"constant_brier_baseline":0.25,"brier_skill":0.44886449364639114,"baseline_adjusted_skill":0.6599999999999999,"baseline_adjusted_score":0.6599999999999999}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB datasets/wiqa / test · target_mass_at_prediction. Population 100 cases / 100 decisions.

Exact perplexity-ai/pplx-decider-flash-v1.2; adapter pplx-decider; settings {"questions_per_call": 1}. Run perplexity-ai__pplx-decider-flash-v1.2.

Published measured S1MB run; community execution/submission status is not assumed independent. Choice is target mass at the selected option; soft targets are not ordinary hard-label accuracy. All published diagnostics: {"target_mass_at_prediction":0.51,"uniform_accuracy_baseline":0.33333333333333326,"fixed_answer_accuracy_baseline":0.37,"baseline_adjusted_skill":0.22222222222222224,"baseline_adjusted_score":0.22222222222222224}. Counts: {"cases": 100, "expected": 100, "succeeded": 100, "failed": 0, "abstained": 0}. Parameter-count conventions excluding lookup/embedding parameters are not model MoE active counts. No executed weight SHA is stated in this display record; exact named repository and adapter are retained.

Pinned public display snapshot 2026-10-17T12:53:13.764Z. Benchmark definitions keep dataset/split and task semantics. Native binary Brier and normalized expected-score MAE are errors, not percents. Failed, partial or missing results never become zero.

Checked / reported: 2026-10-18 · Samples: 100

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB english-v1 · BordaScore

Published english-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 137 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB english-v1 · TaskAvg

Published english-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 137 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB english-v1 · Noul

Published english-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 137 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB english-v1 · Choice

Published english-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 137 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB english-v1 · Score

Published english-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 137 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB english-v1 · GeneralNoul

Published english-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 137 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB english-v1 · GeneralChoice

Published english-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 137 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB english-v1 · GeneralScore

Published english-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 137 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB japanese-v1 · BordaScore

Published japanese-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 27 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB japanese-v1 · TaskAvg

Published japanese-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 27 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB japanese-v1 · Noul

Published japanese-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 27 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB japanese-v1 · Choice

Published japanese-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 27 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB japanese-v1 · Score

Published japanese-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 27 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB japanese-v1 · GeneralNoul

Published japanese-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 27 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB japanese-v1 · GeneralChoice

Published japanese-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 27 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB japanese-v1 · GeneralScore

Published japanese-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 27 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB vision-v1 · BordaScore

Published vision-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 20 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB vision-v1 · TaskAvg

Published vision-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 20 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB vision-v1 · Noul

Published vision-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 20 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB vision-v1 · Choice

Published vision-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 20 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB vision-v1 · Score

Published vision-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 20 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB vision-v1 · GeneralNoul

Published vision-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 20 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB vision-v1 · GeneralChoice

Published vision-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 20 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

S1MB vision-v1 · GeneralScore

Published vision-v1 UI scope; Perplexity Decider Flash 1.2perplexity-ai/pplx-decider-flash-v1.2.

Coverage 20 benchmarks. Read the source scoring specification for adjustment, clipping and scope exclusions.

Value read from the publisher’s rendered table, not recomputed from rounded components. Noul/Choice/Score and General columns are source baseline-adjusted scores. Borda is relative to the complete model cohort in this dated scope; TaskAvg is equal task weight after equal benchmark weight. Neither is native accuracy/Brier/MAE.

Checked / reported: 2026-10-18 · Samples: Not published

Snapshot: 2026-10-17T12:53:13.764Z · Revision: 706ef6637478b78e908be90088f68d8c8f0d52e4 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; All-case accuracy.

Named source route perplexity/pplx-decider-flash-v1.2; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 56

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Hard-case accuracy.

Named source route perplexity/pplx-decider-flash-v1.2; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 19

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Public-case accuracy.

Named source route perplexity/pplx-decider-flash-v1.2; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 14

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Hidden-case accuracy.

Named source route perplexity/pplx-decider-flash-v1.2; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 42

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified

Exact named hosted API observation; Brier-based confidence score.

Named source route perplexity/pplx-decider-flash-v1.2; ModelTested runner v1; tested_on=openrouter. Exact backend weight revision and API inference precision unpublished.

Publisher generated 2026-10-18. Execution dates, token usage and hashes remain in its per-run receipts. No private prompt/response archive imported.

56 authored situations, including 19 hard cases, each with two typed questions; 14 public and 42 hidden cases. Source aggregate averages per-case decision accuracy. Confidence score is 100×(1−mean multiclass Brier/2), not ECE or a calibration probability. Small single-run benchmark; current HF access/size does not inherit this evaluator’s stale closed/API-only metadata. No tariff inferred from rounded cost 0.0.

Checked / reported: 2026-10-18 · Samples: 56

Snapshot: 2026-10-18 · Revision: 081e8f37130a4a9077b8cffe62827d4c3b9e3b23 · Published aggregate observations; result reuse terms not separately specified