ExamenosStart with a focused judgment
A decision model receives context and a defined set of answers. For example, supply a support ticket and ask whether Billing, Technical or Sales should handle it. The model returns probabilities for those choices. Your application decides whether to act or ask for review.
Open the Decisions dashboard to inspect published results. Select up to five systems and use Compare to keep the same choices and filters in a shared link on desktop or phone.
Choose the task before the model
The benchmark selector keeps each publisher, version, population and execution method separate. A multiple-choice reasoning result is different from a routing score or a vision test. Use Find a benchmark to narrow the evaluations, then pin the columns that matter. A dash means no recorded observation, never zero.
Use accuracy and probability quality together
Accuracy describes selected answers. Calibration describes how confidence relates to observed correctness. A model can choose accurately while being overconfident. Brier and log-loss-style metrics evaluate probability quality; their precise definitions matter. Different studies' calibration numbers do not automatically share a scale.
Typed Decisions uses teacher-produced probability labels; its accuracy is teacher agreement, not a guarantee of objective truth. JevBench publishes separate task and question-type metrics. Inspect coverage, refusals, errors, prompt wording and option-order sensitivity when the source reports them.
Read the source labels
Independent results come from the named evaluator. Official and peer-vendor runs have their own columns. Evaluator-published results can include submitted public runs and evaluator-run private tests; their setup disclosure identifies this. A vendor running an independent harness does not make that run independent.
The AA Decision Index and the community Decision Index are different evaluations. Full and public-only editions remain distinct. Examenos does not combine these into a new overall ranking.
OneJev's public-index figures are vendor runs and have their own labeled column. Perplexity's joint-request and single-question scores are separate configurations. On S1MB, native Brier and MAE losses differ from its adjusted scores and cohort-relative Borda ranking. ModelTested uses a small 56-case suite; its Brier-based confidence score is an index, not a percentage calibration error. Open the setup notes before comparing these quantities.
Compare full requests and their actual billing
Decision systems may generate no output tokens while still charging for input. Some providers repeat the context and images for each question. An input tariff is different from a study's cost per million tasks. Local GPU estimates are different again.
Latency needs provider, hardware, request size, question count and concurrency. Hosted measurements can include a gateway and network travel. A local measurement excludes that path. Best-value markers are withheld when the published setups do not support a common ranking.
Keep checkpoint and API identities exact
Jev 1.14 preview does not inherit Jev 1.13 studies. Hosted d1, d1-ultra and open d1 checkpoints stay separate. GPT-6 Luna's Decisions endpoint has its own interface and billing, while Granite Switch's Decision adapter is a particular configuration of a hybrid model. Model profiles retain the exact identity, source links and every recorded result.
Sources: JevBench methods, DecideBench methods, Typed Decisions, OpenAI Decisions documentation, and Liquid billing explanation.
Sources and further reading
These explanations describe Examenos's data and display policy. Numeric examples are available in the linked model/benchmark records, each with original citations. Examenos curates results rather than conducting these evaluations.