Examenos

Official vs independent LLM benchmarks

Understand vendor announcements, peer comparison tables and independent evaluator scores, and why Examenos keeps these sources separate.

Three kinds of evidence

An own-vendor result comes from the model provider's announcement, card, report or documentation. A peer-vendor result appears in another provider's comparison table. Independent results come from an evaluator running its own process. These source relationships are recorded separately.

Similar names do not make identical measurements

A vendor and an evaluator may publish numbers with the same benchmark name while using different tooling, settings or model snapshots. Examenos keeps vendor results in the official section and evaluator results in the independent section. Those values are not merged into one supposedly definitive score.

Compare the source dates, reasoning effort and harness notes before drawing a conclusion. If the source does not state a setting, its absence remains a limitation rather than an invitation to guess.

What the provenance labels mean

Own vendor identifies first-party model sources. Peer vendor identifies comparison spillover. Third-party identifies secondary reporting. Independent tables name the evaluator; they do not silently reuse a vendor's value to fill an evaluator gap.

How to inspect the distinction

On a model profile, read the separate official and independent tables, then expand the record details. The score links point to the source actually used. For evaluator-only comparisons, open ARC Prize's recorded ARC-AGI-2 results and follow the source pages.

Our methodology explains how conflicting records and headline alternatives are retained. Source type is context for assessing evidence, not a guarantee that any particular result generalizes to your workload.

Sources and further reading

These explanations describe Examenos's data and display policy. Numeric examples are available in the linked model/benchmark records, each with original citations. Examenos curates results rather than conducting these evaluations.