Examenos

62% · raw 62 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 0; as reported by Mistral. Harness: Artificial Analysis private prelaunch evaluation; peer harnesses: Claude Code (Qwen), Cloak (DeepSeek), OpenCode (GLM), Kimi Code CLI (Kimi). Demoted 2026-10-06: precise own launch text from https://mistral.ai/news/mistral-large-4 takes precedence.

61.7% · raw 61.7 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

AA private prelaunch evaluation; article is more precise than chart 62%.

28.3% · raw 28.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 1; as reported by Mistral. Harness: Artificial Analysis private prelaunch evaluation. Summary figure; conflicts with detailed chart for some peers; do not average.

28% · raw 28 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 10; as reported by Mistral. Harness: AA private prelaunch evaluation; detailed chart differs from summary chart (Kimi 21 vs 12.9, GLM 40 vs 41.9). Configuration is unspecified; retain discrepancy. Alternative evidence; existing headline retained under source/configuration precedence. No averaging.

50% · raw 50 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 2; as reported by Mistral. Successes, not safety blocks; third-party evaluation reported by Mistral.

59.9% · raw 59.9 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 3; as reported by Mistral. 657 business workflows; AA harness; version not stated. Distinct from public split and v1.0.6.

54.7% · raw 54.7 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 4; as reported by Mistral. Vendor-reported Vals AI result; independently sourced precise results remain in evaluator columns.

15.8% · raw 15.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 5; as reported by Mistral. Vendor-reported Vals AI HLAB; independently sourced precise results remain in evaluator columns.

42% · raw 42 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 6; as reported by Mistral. Bounding-box visual grounding. Article rounds GPT-6 Astra to 41%; use figure's 41.5%.

82% · raw 82 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 7; as reported by Mistral. Vulnerability reproduction and patching; distinct from ordinary CyberGym.

93% · raw 93 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 9; as reported by Mistral. 40 security-competition challenges; rounded vendor figure.

59% · raw 59 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 11; as reported by Mistral. Harness: AA private prelaunch evaluation; source does not establish equivalence to Codebase QnA launch split. Demoted 2026-10-06: precise own launch text from https://mistral.ai/news/mistral-large-4 takes precedence.

59.4% · raw 59.4 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

AA private prelaunch evaluation; article is more precise than chart 59%.

63.1% · raw 63.1 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 13; as reported by Mistral. Chart question answering; exact Pro revision.

18.6% · raw 18.6 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 14; as reported by Mistral. AA all-pass document benchmark reported in Mistral launch.

91.8% · raw 91.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 15; as reported by Mistral. Verified scientific-code workflows, pass@1, n=6; distinct from original SciCode.

67.4% · raw 67.4 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 17; as reported by Mistral. Spreadsheet creation/editing for finance/accounting; vendor-reported Finch.

93.3% · raw 93.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 18; as reported by Mistral. Lakera B3 indirect prompt-injection attack resistance; linked weak dataset.

95.3% · raw 95.3 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 19; as reported by Mistral. Average refusal across JailbreakBench, StrongREJECT and AgentHarm; descriptive refusal rate, not cyber capability.

1.7 score · raw 1.7 score

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 20; as reported by Mistral. 0–2 aggregate, NOT 0–1 or percent. Chart rounds to one decimal; article states Mistral 1.691. Demoted 2026-10-06: precise own launch text from https://mistral.ai/news/mistral-large-4 takes precedence.

1.7 score · raw 1.691 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Native 0–2 scale; article is more precise than rounded chart 1.7.

68% · raw 68 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 21; as reported by Mistral. Internal expert human evaluation, weighted preference against GLM-5.3 only. Detailed STEM figure reports 69%, retained as alternative.

69% · raw 69 %

Alternative · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 16; as reported by Mistral. Detailed STEM weighted win rate; overview reports 68%. Category shares: much better 7%, better 43%, slightly better 15%, tie 22%, slightly worse 14%; rounded shares, do not recompute. Alternative evidence; existing headline retained under source/configuration precedence. No averaging.

62% · raw 62 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 21; as reported by Mistral. Internal expert human evaluation; weighted preference against GLM-5.3 only.

50% · raw 50 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 21; as reported by Mistral. Internal expert human evaluation; weighted preference against GLM-5.3 only.

48% · raw 48 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

Mistral launch figure 21; as reported by Mistral. Internal expert human evaluation; weighted preference against GLM-5.3 only.

49.8% · raw 49.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

As reported by Mistral. AA private prelaunch harness aggregate, as reported by Mistral; separate from evaluator source.

1393 · raw 1393 Elo

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

As reported by Mistral. Long-horizon knowledge work, vendor-rounded Elo; separate from independently sourced AA value.

3.7 score · raw 3.74 score

Headline · unknown effort · Own vendor

Source/record date: 2026-10-06

As reported by Mistral. Blind professional annotator rating on 1–5 scale, five models; vendor-published Surge study, not a universal ranking.

38.4 · raw 38.3770408917494 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

Published non-estimated AA index; exact preview. Reasoning enabled, effort not specified.

$1.13 · raw 1.1317590533008346 USD

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

Evaluator observed cost/task; not API list price.

116.1 tok/s · raw 116.080085205733 tok/s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA median output speed via Mistral; independent from OpenRouter speed snapshot.

1392.5 · raw 1392.53 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA exact preview; Native published scale.

1423.9 · raw 1423.91 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA exact preview; Native published scale.

59.9% · raw 59.901362468087 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA exact preview; Published raw 0–1 fraction converted to percent.

26.8% · raw 26.767676767677 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA exact preview; Published raw 0–1 fraction converted to percent.

54.2% · raw 54.166666666667 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA exact preview; Published raw 0–1 fraction converted to percent.

35% · raw 35.032437442076 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA exact preview; Published raw 0–1 fraction converted to percent.

18.6% · raw 18.6 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA exact preview; Published raw 0–1 fraction converted to percent.

10.6% · raw 10.571428571429 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA exact preview; Published raw 0–1 fraction converted to percent.

-5.3 score · raw -5.3 score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA exact preview; Native published scale.

81.3% · raw 81.333333333333 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA exact preview; Published raw 0–1 fraction converted to percent.

76.4% · raw 76.416184971098 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

Published raw 0–1 fraction converted to percent; accuracy and hallucination are separate metrics.

25.8% · raw 25.816666666667 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

Published raw 0–1 fraction converted to percent; accuracy and hallucination are separate metrics.

41.9% · raw 41.945630195462 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

Published raw 0–1 fraction converted to percent; accuracy and hallucination are separate metrics.

38.3 score · raw 38.262600132251 score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

Published AA industry index; native index scale, not percent.

42.2 score · raw 42.2092826635119 score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

Published AA industry index; native index scale, not percent.

37 score · raw 36.9687229783312 score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

Published AA industry index; native index scale, not percent.

37.6 score · raw 37.5808314171772 score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

Published AA industry index; native index scale, not percent.

43.6 score · raw 43.5568947713933 score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

Published AA industry index; native index scale, not percent.

45.2% · raw 45.151515151515 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

Published raw 0–1 fraction converted to percent.

1515.4 · raw 1515.38 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA-Briefcase component; native Elo.

1354.5 · raw 1354.54 Elo

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-06

AA-Briefcase component; native Elo.

48% · raw 48.045 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-06

Vals Index v2.1; GDP-weighted finance/coding/legal/tax. Published accuracy already percent; no conversion.

78.4% · raw 78.403 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-06

Vals primary standalone benchmark; accuracy already percent. Provider: Mistral AI; high effort, temperature 1, top_p 0.95, max output 256000. Harness: OpenHands.

54.7% · raw 54.678 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-06

Vals primary standalone benchmark; accuracy already percent. Provider: Mistral AI; high effort, temperature 1, top_p 0.95, max output 256000.

56.2% · raw 56.213 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-06

Vals primary standalone benchmark; accuracy already percent. Provider: Mistral AI; high effort, temperature 1, top_p 0.95, max output 256000.

22.7% · raw 22.727 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-06

Vals primary standalone benchmark; accuracy already percent. Provider: Mistral AI; high effort, temperature 1, top_p 0.95, max output 256000.

30.6% · raw 30.561 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-06

Vals primary standalone benchmark; accuracy already percent. Provider: Mistral AI; high effort, temperature 1, top_p 0.95, max output 256000. Full standalone benchmark, not Index subset (22.799%).

31.7% · raw 31.731 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-06

Vals primary standalone benchmark; accuracy already percent. Provider: Mistral AI; high effort, temperature 1, top_p 0.95, max output 256000.

15.8% · raw 15.833 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-06

Vals primary standalone benchmark; accuracy already percent. Provider: Mistral AI; high effort, temperature 1, top_p 0.95, max output 256000.

63.3% · raw 63.294 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-06

Vals primary standalone benchmark; accuracy already percent. Provider: Mistral AI; high effort, temperature 1, top_p 0.95, max output 256000.

22.8% · raw 22.799 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-06

Index subset: 50/120 CLI tasks plus 10 COBOL tasks, weighted 75%/25%; distinct from full standalone benchmark. Published percent.