Examenos

15.6% · raw 15.56 %

Alternative · medium effort · Own vendor

Source/record date: 2026-10-27

Cognition primary public data.json (https://cognition.com/data/frontier-code-2.0/data.json), FrontierCode 2.0 main, 132 tasks; weighted new_score x100, not unweighted correct. Harness: devin. {"correct": 0.176, "tokens": 28705, "cost": 0.442, "flagged_rate": 0.0197}. Cost is source-observed mean USD per rollout at list API pricing, not an API rate. Data checked 5 November; evaluation date unpublished. Headline is the best-scoring published effort, as Cognition displays it.

23.1% · raw 23.06 %

Alternative · high effort · Own vendor

Source/record date: 2026-10-27

Cognition primary public data.json (https://cognition.com/data/frontier-code-2.0/data.json), FrontierCode 2.0 main, 132 tasks; weighted new_score x100, not unweighted correct. Harness: devin. {"correct": 0.2595, "tokens": 68546, "cost": 0.958, "flagged_rate": 0.0288}. Cost is source-observed mean USD per rollout at list API pricing, not an API rate. Data checked 5 November; evaluation date unpublished. Headline is the best-scoring published effort, as Cognition displays it.

23.8% · raw 23.78 %

Headline · max effort · Own vendor

Source/record date: 2026-10-27

Cognition primary public data.json (https://cognition.com/data/frontier-code-2.0/data.json), FrontierCode 2.0 main, 132 tasks; weighted new_score x100, not unweighted correct. Harness: devin. {"correct": 0.2667, "tokens": 97584, "cost": 1.261, "flagged_rate": 0.0242}. Cost is source-observed mean USD per rollout at list API pricing, not an API rate. Data checked 5 November; evaluation date unpublished. Headline is the best-scoring published effort, as Cognition displays it.

43.1% · raw 43.09 %

Alternative · medium effort · Own vendor

Source/record date: 2026-09-10

Cognition primary public data.json (https://cognition.com/data/frontiercode-leaderboard/data.json, key v1_1), FrontierCode 1.1 main; weighted new_score x100, not unweighted correct. Harness: devin. {"correct": 0.4833, "tokens": 25701.5, "cost": 0.3712, "flagged_rate": 0.0}. Cost is source-observed USD per rollout; not an API rate. Data checked 5 November; evaluation date unpublished. Headline is the best-scoring published effort.

47.2% · raw 47.2 %

Alternative · high effort · Own vendor

Source/record date: 2026-09-10

Cognition primary public data.json (https://cognition.com/data/frontiercode-leaderboard/data.json, key v1_1), FrontierCode 1.1 main; weighted new_score x100, not unweighted correct. Harness: devin. {"correct": 0.5235, "tokens": 52821.9, "cost": 0.7813, "flagged_rate": 0.0}. Cost is source-observed USD per rollout; not an API rate. Data checked 5 November; evaluation date unpublished. Headline is the best-scoring published effort.

50% · raw 50 %

Headline · max effort · Own vendor

Source/record date: 2026-09-10

Cognition primary public data.json (https://cognition.com/data/frontiercode-leaderboard/data.json, key v1_1), FrontierCode 1.1 main; weighted new_score x100, not unweighted correct. Harness: devin. {"correct": 0.5547, "tokens": 72764.7, "cost": 1.1761, "flagged_rate": 0.0}. Cost is source-observed USD per rollout; not an API rate. Data checked 5 November; evaluation date unpublished. Headline is the best-scoring published effort.

56.4% · raw 56.41 %

Alternative · medium effort · Own vendor

Source/record date: 2026-09-10

Cognition primary public data.json (https://cognition.com/data/frontiercode-leaderboard/data.json, key v1_1), FrontierCode 1.1 extended; weighted new_score x100, not unweighted correct. Harness: devin. {"correct": 0.6222, "tokens": 21337.6, "cost": 0.3041, "flagged_rate": 0.0}. Cost is source-observed USD per rollout; not an API rate. Data checked 5 November; evaluation date unpublished. Headline is the best-scoring published effort.

60.1% · raw 60.08 %

Alternative · high effort · Own vendor

Source/record date: 2026-09-10

Cognition primary public data.json (https://cognition.com/data/frontiercode-leaderboard/data.json, key v1_1), FrontierCode 1.1 extended; weighted new_score x100, not unweighted correct. Harness: devin. {"correct": 0.6563, "tokens": 43469.1, "cost": 0.6357, "flagged_rate": 0.0}. Cost is source-observed USD per rollout; not an API rate. Data checked 5 November; evaluation date unpublished. Headline is the best-scoring published effort.

62.5% · raw 62.46 %

Headline · max effort · Own vendor

Source/record date: 2026-09-10

Cognition primary public data.json (https://cognition.com/data/frontiercode-leaderboard/data.json, key v1_1), FrontierCode 1.1 extended; weighted new_score x100, not unweighted correct. Harness: devin. {"correct": 0.6844, "tokens": 59918.7, "cost": 0.94, "flagged_rate": 0.0}. Cost is source-observed USD per rollout; not an API rate. Data checked 5 November; evaluation date unpublished. Headline is the best-scoring published effort.

73% · raw 73 %

Headline · unknown effort · Own vendor

Source/record date: 2026-09-10

Cognition SWE-2 launch table (10 September 2026), row “DeepSWE 1.1”, column “SWE-2”. Cognition SWE-2 post, Appendix A: a model’s public result where one exists, otherwise Cognition’s own run in the harness it was developed for (Claude Code for Anthropic, Codex for OpenAI, Grok Build for xAI, Devin CLI for open-weight models); best score across reasoning-effort settings. Effort per cell unstated.

92.8% · raw 92.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-09-10

Cognition SWE-2 launch table (10 September 2026), row “Terminal-Bench 2.1”, column “SWE-2”. Cognition SWE-2 post, Appendix A: a model’s public result where one exists, otherwise Cognition’s own run in the harness it was developed for (Claude Code for Anthropic, Codex for OpenAI, Grok Build for xAI, Devin CLI for open-weight models); best score across reasoning-effort settings. Effort per cell unstated.

27.3% · raw 27.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-09-10

Cognition SWE-2 launch table (10 September 2026), row “Terminal-Bench 4.0”, column “SWE-2”. Cognition SWE-2 post, Appendix A: a model’s public result where one exists, otherwise Cognition’s own run in the harness it was developed for (Claude Code for Anthropic, Codex for OpenAI, Grok Build for xAI, Devin CLI for open-weight models); best score across reasoning-effort settings. Effort per cell unstated.

56.3% · raw 56.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Cognition SWE-2 Flash launch table (4 November 2026), row “SWE-Atlas QA”, column “SWE-2”. Cognition labels it “SWE-Atlas QA”. Cognition publishes no methodology for this table; its SWE-2 post says it reports a model’s public result where one exists and otherwise runs it in the harness it was developed for (Claude Code for Anthropic, Codex for OpenAI, Devin CLI for open-weight models) at its best reasoning effort. Effort per cell unstated.

90.6% · raw 90.6 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Cognition SWE-2 Flash launch table (4 November 2026), row “SWE-Bench Pro v2 Hard”, column “SWE-2”. Cognition publishes no methodology for this table; its SWE-2 post says it reports a model’s public result where one exists and otherwise runs it in the harness it was developed for (Claude Code for Anthropic, Codex for OpenAI, Devin CLI for open-weight models) at its best reasoning effort. Effort per cell unstated.

36.1% · raw 36.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-11-04

Cognition SWE-2 Flash launch table (4 November 2026), row “ExploitBench”, column “SWE-2”. Cognition publishes no methodology for this table; its SWE-2 post says it reports a model’s public result where one exists and otherwise runs it in the harness it was developed for (Claude Code for Anthropic, Codex for OpenAI, Devin CLI for open-weight models) at its best reasoning effort. Effort per cell unstated.

99.8% · raw 99.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-09-10

Printed bar label in the figure “Propaganda and Censorship Eval, by language” of Cognition’s SWE-2 post (10 September 2026), read from the rendered page on 5 November 2026 (receipt in research-raw). Judge: GPT 5.6 Luna; binary pass/fail on 145 Pan and Xu questions; empty responses and errors excluded. Effort unstated.

95.2% · raw 95.2 %

Headline · unknown effort · Own vendor

Source/record date: 2026-09-10

Printed bar label in the figure “Propaganda and Censorship Eval, by language” of Cognition’s SWE-2 post (10 September 2026), read from the rendered page on 5 November 2026 (receipt in research-raw). Judge: GPT 5.6 Luna; binary pass/fail on 145 Pan and Xu questions; empty responses and errors excluded. Effort unstated.

99.1% · raw 99.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-09-10

Printed bar label in the figure “Propaganda and Censorship Eval, by language” of Cognition’s SWE-2 post (10 September 2026), read from the rendered page on 5 November 2026 (receipt in research-raw). Judge: GPT 5.6 Luna; binary pass/fail on 145 Pan and Xu questions; empty responses and errors excluded. Effort unstated.

98% · raw 98 %

Headline · unknown effort · Own vendor

Source/record date: 2026-09-10

Cognition SWE-2 post (10 September 2026): “SWE-2 passed 98.0% of attempts overall”. Judge: GPT 5.6 Luna; binary pass/fail on 145 Pan and Xu questions in three languages; empty responses and errors excluded.