Examenos

29.7% · raw 29.7 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “HLE text only”, column “Inkling effort=0.99”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

46% · raw 46 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “HLE with tools”, column “Inkling effort=0.99”. The post attributes Humanity’s Last Exam scores to Artificial Analysis but does not say whether that covers this with-tools row; tools and harness unpublished. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

97.1% · raw 97.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “AIME 2026”, column “Inkling effort=0.99”. Sampling and repeats unpublished. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

87.2% · raw 87.2 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “GPQA Diamond”, column “Inkling effort=0.99”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

77.6% · raw 77.6 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “SWEBench Verified”, column “Inkling effort=0.99”. Bash-only harness for Thinking Machines’ models. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

54.3% · raw 54.3 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “SWEBench Pro Public”, column “Inkling effort=0.99”. Public set; harness unpublished. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

63.8% · raw 63.8 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “Terminal Bench 2.1 Best Harness”, column “Inkling effort=0.99”. Internal coding harness for Thinking Machines’ models; a small number of solutions found contaminated from web search were scored 0. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

56.2% · raw 56.2 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-11-04

Ricursive in-house run (italic in the Ripple-65B Preview card, revision 240d5966cbe0), row “Terminal-Bench 2.1 (Terminus 2)”, column “Inkling 975B”; not a result from the model’s developer. Harness: Terminus 2, the official harness, at 128K sequence length with summarization (card: Ricursive found Terminal-Bench 2.1 scores universally lower in this setup except for Muse Spark Open). Different harness from the developer’s own figure, so non-headline. Ricursive: all models tested at full BF16 precision with maximum effort levels; the peer’s exact effort name is unstated. Non-italic peer values in the table are the developers’ own figures and are not stored. Non-headline: the first-party row (63.8, https://thinkingmachines.ai/news/introducing-inkling/) holds this cell.

1238 · raw 1238 Elo

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “GDPVal-AA v2”, column “Inkling effort=0.99”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

1233 · raw 1233 Elo

Alternative · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Inkling Hugging Face card (revision 828496eeae4c), evaluation table row “GDPVal-AA v2”: 1233, where the launch post prints 1238. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. The card does not explain the difference; kept as an alternative. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Non-headline: the first-party row (1238.0, https://thinkingmachines.ai/news/introducing-inkling/) holds this cell.

76% · raw 76 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “MCP Atlas”, column “Inkling effort=0.99”. All tasks; the Inkling-Small post names Scale AI as the source of its MCP Atlas values. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

74.1% · raw 74.1 %

Alternative · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Inkling Hugging Face card (revision 828496eeae4c), evaluation table row “MCP Atlas”: 74.1%, where the launch post prints 76. All tasks; the Inkling-Small post names Scale AI as the source of its MCP Atlas values. The card does not explain the difference; kept as an alternative. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Non-headline: the first-party row (76.0, https://thinkingmachines.ai/news/introducing-inkling/) holds this cell.

23.7% · raw 23.7 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “Tau 3 Banking”, column “Inkling effort=0.99”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

25% · raw 25 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-05

Harness: Reflection launch evaluation; full agent, revision and token-budget settings not published. As reported by Reflection; peer comparison, not an independent evaluator record. Peer precision and effort unstated. Row “tau3 banking”, column “Inkling”. Thinking Machines’ own value for this cell is 23.7, so 25.0 is a separate run. Non-headline: the first-party row (23.7, https://thinkingmachines.ai/news/introducing-inkling/) holds this cell.

45.5% · raw 45.5 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “Toolathlon Verified”, column “Inkling effort=0.99”. Harness unpublished. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

77.1% · raw 77.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “BrowseComp w/ ctx management”, column “Inkling effort=0.99”. With context management; harness unpublished. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

43.9% · raw 43.9 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “SimpleQA Verified”, column “Inkling effort=0.99”. Grader unpublished. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

2.1 · raw 2.1 index

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “AA Omniscience”, column “Inkling effort=0.99”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Omniscience index (−100 to 100), not an accuracy. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

1 · raw 1 index

Alternative · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Inkling Hugging Face card (revision 828496eeae4c), evaluation table row “AA Omniscience”: 1.0%, where the launch post prints 2.1. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Omniscience index (−100 to 100), not an accuracy. The card does not explain the difference; kept as an alternative. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Non-headline: the first-party row (2.1, https://thinkingmachines.ai/news/introducing-inkling/) holds this cell.

79.8% · raw 79.8 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “IFBench”, column “Inkling effort=0.99”. Prompt or instruction level and strict or loose scoring unpublished. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

88.7% · raw 88.7 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “Global-MMLU-Lite”, column “Inkling effort=0.99”. Prompting unpublished. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

73.5% · raw 73.5 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “MMMU Pro Standard 10”, column “Inkling effort=0.99”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Standard setting with 10 options. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

78.1% · raw 78.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “Charxiv RQ”, column “Inkling effort=0.99”. No tools. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

82% · raw 82 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “Charxiv RQ with python”, column “Inkling effort=0.99”. With a Python tool. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

56.6% · raw 56.6 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “Audio MC”, column “Inkling effort=0.99”. Thinking Machines states it uses Scale AI’s reported score for Audio MultiChallenge (named in the Inkling-Small post). Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

77.2% · raw 77.2 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “MMAU”, column “Inkling effort=0.99”. Split unpublished. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

91.4% · raw 91.4 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “VoiceBench”, column “Inkling effort=0.99”. Thinking Machines added a system message telling models to follow the expected answer format, because VoiceBench grades by string matching. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

78% · raw 78 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “FORTRESS Adversarial”, column “Inkling effort=0.99”. Higher is better: the rate of refusing harmful requests. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

95.9% · raw 95.9 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “FORTRESS Benign”, column “Inkling effort=0.99”. Higher is better: the rate of still answering safe look-alike requests. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

98.6% · raw 98.6 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-15

Thinking Machines’ Introducing Inkling post (15 July 2026), main benchmark table, row “StrongREJECT”, column “Inkling effort=0.99”. Higher is better. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished; the HF card says comparison scores were generated 14 July 2026.

41 · raw 41 index

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-30

Thinking Machines’ Inkling-Small post (30 July 2026), main benchmark table, row “AA Index v4.1”, column “Inkling”: a value the Inkling launch post did not print. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Printed with a % sign; stored as the index value. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished.

46.1% · raw 46.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-30

Thinking Machines’ Inkling-Small post (30 July 2026), main benchmark table, row “SciCode”, column “Inkling”: a value the Inkling launch post did not print. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished.

78.8% · raw 78.8 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-30

Thinking Machines’ Inkling-Small post (30 July 2026), main benchmark table, row “MCP Atlaspublic / all”, column “Inkling”: a value the Inkling launch post did not print. Thinking Machines states it uses Scale AI’s reported score for this benchmark. Public task split. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished.

839 · raw 839 Elo

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-30

Thinking Machines’ Inkling-Small post (30 July 2026), main benchmark table, row “AA-Briefcase”, column “Inkling”: a value the Inkling launch post did not print. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Elo-style rating. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished.

86.3% · raw 86.3 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-30

Thinking Machines’ Inkling-Small post (30 July 2026), main benchmark table, row “HMMT Feb 2026”, column “Inkling”: a value the Inkling launch post did not print. Sampling and repeats unpublished. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished.

5.4% · raw 5.4 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-30

Thinking Machines’ Inkling-Small post (30 July 2026), main benchmark table, row “CritPt”, column “Inkling”: a value the Inkling launch post did not print. Harness unpublished. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished.

79.5% · raw 79.5 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-30

Thinking Machines’ Inkling-Small post (30 July 2026), main benchmark table, row “ARC-AGI-1”, column “Inkling”: a value the Inkling launch post did not print. Thinking Machines states it uses ARC Prize’s reported score (verified semi-private set). Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished.

36.5% · raw 36.5 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-30

Thinking Machines’ Inkling-Small post (30 July 2026), main benchmark table, row “ARC-AGI-2”, column “Inkling”: a value the Inkling launch post did not print. Thinking Machines states it uses ARC Prize’s reported score (verified semi-private set). Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown. Execution date unpublished.

60.1 · raw 60.1 index

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-30

Thinking Machines’ Inkling-Small post (30 July 2026), forecasting table, column “Inkling”: Forecasting Research Institute ForecastBench, no search, Brier Index (higher is better), 60.1 ± 0.54. Thinking Machines names the Forecasting Research Institute and ProphetArena as the sources of these scores; results were obtained during testing between 19 and 28 July 2026. The ± value (0.54) is the post’s published uncertainty, interval type unstated. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown.

61 · raw 61 index

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-30

Thinking Machines’ Inkling-Small post (30 July 2026), forecasting table, column “Inkling”: Forecasting Research Institute ForecastBench, with search, Brier Index (higher is better), 61.0 ± 0.56. Thinking Machines names the Forecasting Research Institute and ProphetArena as the sources of these scores; results were obtained during testing between 19 and 28 July 2026. The ± value (0.56) is the post’s published uncertainty, interval type unstated. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown.

0.1 score · raw 0.1276 score

Headline · xhigh effort · Own vendor

Source/record date: 2026-07-30

Thinking Machines’ Inkling-Small post (30 July 2026), forecasting table, column “Inkling”: Prophet Arena Brier score (lower is better), 0.1276 ± 0.0092. Thinking Machines names the Forecasting Research Institute and ProphetArena as the sources of these scores; results were obtained during testing between 19 and 28 July 2026. The ± value (0.0092) is the post’s published uncertainty, interval type unstated. Thinking Machines runs its own models at effort 0.99 (a 0–1 setting) and temperature 1.0; Effort 0.99 is stored as xhigh: Thinking Machines’ Inkling 2 Small post (5 November 2026, https://thinkingmachines.ai/news/inkling-2-small/) names this setting “effort 0.99 (xhigh)”; relabelled 7 November 2026 from unknown.

91.3% · raw 91.3 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-11-04

Ricursive in-house run (italic in the Ripple-65B Preview card, revision 240d5966cbe0), row “MMLU”, column “Inkling 975B”; not a result from the model’s developer. Ricursive: all models tested at full BF16 precision with maximum effort levels; the peer’s exact effort name is unstated. Non-italic peer values in the table are the developers’ own figures and are not stored.

87.3% · raw 87.3 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-11-04

Ricursive in-house run (italic in the Ripple-65B Preview card, revision 240d5966cbe0), row “MMLU-Pro”, column “Inkling 975B”; not a result from the model’s developer. Ricursive: all models tested at full BF16 precision with maximum effort levels; the peer’s exact effort name is unstated. Non-italic peer values in the table are the developers’ own figures and are not stored.

97.5% · raw 97.5 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-11-04

Ricursive in-house run (italic in the Ripple-65B Preview card, revision 240d5966cbe0), row “AIME 2024”, column “Inkling 975B”; not a result from the model’s developer. Ricursive: all models tested at full BF16 precision with maximum effort levels; the peer’s exact effort name is unstated. Non-italic peer values in the table are the developers’ own figures and are not stored.

91.7% · raw 91.7 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-11-04

Ricursive in-house run (italic in the Ripple-65B Preview card, revision 240d5966cbe0), row “AIME 2025”, column “Inkling 975B”; not a result from the model’s developer. Ricursive: all models tested at full BF16 precision with maximum effort levels; the peer’s exact effort name is unstated. Non-italic peer values in the table are the developers’ own figures and are not stored.

2% · raw 2 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-11-04

Ricursive in-house run (italic in the Ripple-65B Preview card, revision 240d5966cbe0), row “SWE-bench Pro v2.0 Hard”, column “Inkling 975B”; not a result from the model’s developer. Harness: OpenCode (the card says Ricursive uses OpenCode wherever no harness is named). The release blog’s tooltip for the same cell gives a separate “best harness” run. Ricursive: all models tested at full BF16 precision with maximum effort levels; the peer’s exact effort name is unstated. Non-italic peer values in the table are the developers’ own figures and are not stored.

39.6% · raw 39.6 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-11-04

Ricursive release blog bar chart “SWE-bench Pro v2.0 Hard”, Inkling bar; tooltip “Best harness / Evaluated in-house” (read on 5 November 2026; receipt in research-raw). Not a result from the model’s developer. Harness: Ricursive’s best harness for this model, not named; the card’s 2.0 for the same cell is its OpenCode run. Ricursive: all models tested at full BF16 precision with maximum effort levels. Chart value, so the card’s table value keeps the cell. Non-headline: the spillover row (2.0, https://huggingface.co/ricursiveAI/Ripple-65B-Preview/blob/240d5966cbe00f0627c699e59ef58560b60303cf/README.md) holds this cell.

1.8% · raw 1.8 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-11-04

Ricursive in-house run (italic in the Ripple-65B Preview card, revision 240d5966cbe0), row “DeepSWE 1.1”, column “Inkling 975B”; not a result from the model’s developer. Harness: OpenCode (the card says Ricursive uses OpenCode wherever no harness is named). The release blog’s tooltip for the same cell gives a separate “best harness” run. Ricursive: all models tested at full BF16 precision with maximum effort levels; the peer’s exact effort name is unstated. Non-italic peer values in the table are the developers’ own figures and are not stored.

5% · raw 5 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-11-04

Ricursive release blog bar chart “DeepSWE 1.1”, Inkling bar; tooltip “Best harness / Evaluated in-house” (read on 5 November 2026; receipt in research-raw). Not a result from the model’s developer. Harness: Ricursive’s best harness for this model, not named; the card’s 1.8 for the same cell is its OpenCode run. Ricursive: all models tested at full BF16 precision with maximum effort levels. Chart value, so the card’s table value keeps the cell. Non-headline: the spillover row (1.8, https://huggingface.co/ricursiveAI/Ripple-65B-Preview/blob/240d5966cbe00f0627c699e59ef58560b60303cf/README.md) holds this cell.

14.6% · raw 14.6 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-11-04

Ricursive in-house run (italic in the Ripple-65B Preview card, revision 240d5966cbe0), row “AutomationBench (Public)”, column “Inkling 975B”; not a result from the model’s developer. Harness: OpenCode (the card says Ricursive uses OpenCode wherever no harness is named). Ricursive: all models tested at full BF16 precision with maximum effort levels; the peer’s exact effort name is unstated. Non-italic peer values in the table are the developers’ own figures and are not stored.

78.3% · raw 78.3 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-11-04

Ricursive in-house run (italic in the Ripple-65B Preview card, revision 240d5966cbe0), row “AA-LCR”, column “Inkling 975B”; not a result from the model’s developer. Version unstated. Ricursive: all models tested at full BF16 precision with maximum effort levels; the peer’s exact effort name is unstated. Non-italic peer values in the table are the developers’ own figures and are not stored.

77.3% · raw 77.3 %

Alternative · unknown effort · Peer vendor

Source/record date: 2026-10-05

Harness: Reflection launch evaluation; full agent, revision and token-budget settings not published. As reported by Reflection; peer comparison, not an independent evaluator record. Peer precision and effort unstated. Row “AA-LCR”, column “Inkling”. Non-headline: the spillover row (78.3, https://huggingface.co/ricursiveAI/Ripple-65B-Preview/blob/240d5966cbe00f0627c699e59ef58560b60303cf/README.md) holds this cell.

14.2 · raw 14.2 index

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-05

Harness: Reflection launch evaluation; full agent, revision and token-budget settings not published. As reported by Reflection; peer comparison, not an independent evaluator record. Peer precision and effort unstated. Row “AA Omniscience (index) - public split only (our results)”, column “Inkling”. Reflection marks the public-split Omniscience row as its own results.

40.1% · raw 40.1 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-15

Harness: GUI-only, 361 tasks, 200 steps, partial reward mean over four attempts. Meta internal run of the peer; do not assume identical run budgets. Meta labels the peer “Inkling (xhigh)”; Thinking Machines’ effort is a 0–1 setting, so effort stays unknown. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

14% · raw 14 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-15

Harness: 100 questions, pass@5 (at least one correct of five), not pass@1. Tools enabled; GPT-OSS-120B high/container judge. Meta internal run of the peer. Meta labels the peer “Inkling (xhigh)”; Thinking Machines’ effort is a 0–1 setting, so effort stays unknown. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

34.2% · raw 34.2 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-15

Harness: 388 questions (135 multiple-choice/253 free-form); tools enabled, GPT-OSS-120B high judge. Meta internal run of the peer. Meta labels the peer “Inkling (xhigh)”; Thinking Machines’ effort is a 0–1 setting, so effort stays unknown. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

52.3% · raw 52.3 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-15

Harness: Tools enabled, 2,024 bilingual questions, five-run mean; Meta internal runs. Meta labels the peer “Inkling (xhigh)”; Thinking Machines’ effort is a 0–1 setting, so effort stays unknown. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

18% · raw 18 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-15

Harness: Tools enabled; 3,000-question/eight-category subset excluding Notable People, GPT-OSS-120B high judge. Meta internal run of the peer. Meta labels the peer “Inkling (xhigh)”; Thinking Machines’ effort is a 0–1 setting, so effort stays unknown. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

55.2% · raw 55.2 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-15

Harness: Official main 1,012-video violation-of-expectation accuracy; Meta internal runs. Meta labels the peer “Inkling (xhigh)”; Thinking Machines’ effort is a 0–1 setting, so effort stays unknown. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

36.9% · raw 36.9 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-15

Harness: 0-shot full 3,200 questions/800 videos, grouped scoring, subtitles in prompt/no audio; Meta internal runs. Meta labels the peer “Inkling (xhigh)”; Thinking Machines’ effort is a 0–1 setting, so effort stays unknown. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

86.4% · raw 86.4 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-15

Harness: Internal 202 behaviors, five attack conditions; mean union-broken ASRs with/without developer prompts; lower is better. Meta labels the peer “Inkling (xhigh)”; Thinking Machines’ effort is a 0–1 setting, so effort stays unknown. As reported by Meta in Muse Spark Open release, not direct evaluator verification. Source checked 15 October 2026; exact execution date unpublished.

78.6% · raw 78.6 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Abuse Entrapment & Coercive Control (“Grooming”)”.

63.5% · raw 63.5 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Biological Weapons”.

79.7% · raw 79.7 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Chemical Weapons”.

85% · raw 85 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Child Safety (Non-CSAM)”.

80% · raw 80 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Copyright, Protected-Text Recitation, and Memorized Source Continuation”.

80.2% · raw 80.2 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Cyber Offense”.

79.1% · raw 79.1 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Fraud & Deception”.

90.5% · raw 90.5 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Hate Speech & Harassment”.

80.8% · raw 80.8 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Illegal Activities”.

78.1% · raw 78.1 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Privacy Violations”.

65.5% · raw 65.5 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Radiological and Nuclear Weapons”.

76.4% · raw 76.4 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Self-Harm & Suicide”.

81.3% · raw 81.3 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Sex-Related Crimes (Non-CSAM)”.

87.5% · raw 87.5 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Sexual Content (Adult, Non-CSAM)”.

80.5% · raw 80.5 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harm category “Violence, Violent Crimes, Threats, and Incitement”.

2.1% · raw 2.1 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harmless-request behaviour breakdown.

0.8% · raw 0.8 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harmless-request behaviour breakdown.

1% · raw 1 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harmless-request behaviour breakdown.

96% · raw 96 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Harmless-request behaviour breakdown.

0% · raw 0 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Agentic harmful-compliance rate; lower is better.

37% · raw 37 %

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-16

Reflection Beam Safety Technical Report, 16 October release. Vendor-reported evaluation, including commissioned external work, not a directly verified independent board. Model execution dates and unstated precision/effort unknown. Source-specific population/harness; not a general safety guarantee. As reported by Reflection; exact peer label Inkling 1T. Reflection’s label “Inkling 1T” rounds Inkling’s 975B total parameters; it is the only Thinking Machines model of that size, so the row sits on Inkling. Agentic harmful-compliance rate; lower is better.

14% · raw 14 %

Headline · unknown effort · Own vendor

Source/record date: 2026-07-15

Cognition primary public data.json (https://cognition.com/data/frontiercode-leaderboard/data.json, key v1_1), FrontierCode 1.1 main; weighted new_score x100, not unweighted correct. Harness: mini-swe-agent. {"correct": 0.159, "tokens": 33610.8, "cost": 3.6, "flagged_rate": 0.076}. Cost is source-observed USD per rollout; not an API rate. Data checked 5 November; evaluation date unpublished. Board effort label “0.99” (Thinking Machines’ 0–1 setting), stored as effort unknown.

24.8% · raw 24.8 %

Headline · unknown effort · Own vendor

Source/record date: 2026-07-15

Cognition primary public data.json (https://cognition.com/data/frontiercode-leaderboard/data.json, key v1_1), FrontierCode 1.1 extended; weighted new_score x100, not unweighted correct. Harness: mini-swe-agent. {"correct": 0.278, "tokens": 30149.8, "cost": 3.2, "flagged_rate": 0.0727}. Cost is source-observed USD per rollout; not an API rate. Data checked 5 November; evaluation date unpublished. Board effort label “0.99” (Thinking Machines’ 0–1 setting), stored as effort unknown.

0.318 bpb · raw 0.318 bpb

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-27

Harness: Mirendil vendor test on its own 42.9 MB repository-code stream. Window 32,768 bytes, stride W/8; mean scored context 30,709 bytes. Value preserves the publisher data table’s displayed three decimal places. As reported by Mirendil; source describes this peer as post-trained. Exact checkpoint, generation/runtime and execution date unpublished. Lower bits per byte is better; not interchangeable with an agentic benchmark.

0.262 bpb · raw 0.262 bpb

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-27

Harness: Mirendil vendor test on its own 42.9 MB repository-code stream. Window 65,536 bytes, stride W/8; mean scored context 61,396 bytes. Value preserves the publisher data table’s displayed three decimal places. As reported by Mirendil; source describes this peer as post-trained. Exact checkpoint, generation/runtime and execution date unpublished. Lower bits per byte is better; not interchangeable with an agentic benchmark.

0.240 bpb · raw 0.24 bpb

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-27

Harness: Mirendil vendor test on its own 42.9 MB repository-code stream. Window 131,072 bytes, stride W/8; mean scored context 122,705 bytes. Value preserves the publisher data table’s displayed three decimal places. As reported by Mirendil; source describes this peer as post-trained. Exact checkpoint, generation/runtime and execution date unpublished. Lower bits per byte is better; not interchangeable with an agentic benchmark.

0.231 bpb · raw 0.231 bpb

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-27

Harness: Mirendil vendor test on its own 42.9 MB repository-code stream. Window 262,144 bytes, stride W/8; mean scored context 245,059 bytes. Value preserves the publisher data table’s displayed three decimal places. As reported by Mirendil; source describes this peer as post-trained. Exact checkpoint, generation/runtime and execution date unpublished. Lower bits per byte is better; not interchangeable with an agentic benchmark.

0.227 bpb · raw 0.227 bpb

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-27

Harness: Mirendil vendor test on its own 42.9 MB repository-code stream. Window 524,288 bytes, stride W/8; mean scored context 488,714 bytes. Value preserves the publisher data table’s displayed three decimal places. As reported by Mirendil; source describes this peer as post-trained. Exact checkpoint, generation/runtime and execution date unpublished. Lower bits per byte is better; not interchangeable with an agentic benchmark.

0.229 bpb · raw 0.229 bpb

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-27

Harness: Mirendil vendor test on its own 42.9 MB repository-code stream. Window 1,048,576 bytes, stride W/8; mean scored context 971,816 bytes. Value preserves the publisher data table’s displayed three decimal places. As reported by Mirendil; source describes this peer as post-trained. Exact checkpoint, generation/runtime and execution date unpublished. Lower bits per byte is better; not interchangeable with an agentic benchmark.

0.234 bpb · raw 0.234 bpb

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-27

Harness: Mirendil vendor test on its own 42.9 MB repository-code stream. Window 2,097,152 bytes, stride W/8; mean scored context 1,921,185 bytes. Value preserves the publisher data table’s displayed three decimal places. As reported by Mirendil; source describes this peer as post-trained. Exact checkpoint, generation/runtime and execution date unpublished. Lower bits per byte is better; not interchangeable with an agentic benchmark.

0.239 bpb · raw 0.239 bpb

Headline · unknown effort · Peer vendor

Source/record date: 2026-10-27

Harness: Mirendil vendor test on its own 42.9 MB repository-code stream. Window 3,670,016 bytes, stride W/8; mean scored context 3,303,148 bytes. Value preserves the publisher data table’s displayed three decimal places. As reported by Mirendil; source describes this peer as post-trained. Exact checkpoint, generation/runtime and execution date unpublished. Lower bits per byte is better; not interchangeable with an agentic benchmark.

23.8% · raw 23.8 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-02

pass^5, the share of 80 tasks solved on all 5 attempts; not pass@1. AA's only Inkling entry; Thinking Machines exposes effort as a 0–1 setting and AA labels this configuration xhigh. Manual observation of Artificial Analysis's public AA-AnalystAgent evaluation page, "AA-AnalystAgent: Score" chart, 2 November 2026 (all 51 listed configurations selected; rendered bar label paired with the x-axis model link; research-raw/aa-analyst-mlcr-2026-11-02/cap_analystagent_score.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date. Added 2026-11-05 when the model was added to Examenos.

6.1% · raw 6.1 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-02

Pass rate on AA's private held-out expert and compound set, 60 questions run 3 times each. AA's only Inkling entry; Thinking Machines exposes effort as a 0–1 setting and AA labels this configuration xhigh. Manual observation of Artificial Analysis's public MLCR-AA evaluation page, "MLCR-AA: Score" chart, 2 November 2026 (all 176 listed configurations selected; rendered bar label paired with the x-axis model link; research-raw/aa-analyst-mlcr-2026-11-02/cap_mlcr_score.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date. Added 2026-11-05 when the model was added to Examenos.

12 · raw 12 index

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

Intelligence Index v5.0.1 (AA changelog, 3 November 2026: v5.0 with CritPt v2 regraded); never compared with v5.0 or v4.3.2 rows. AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public Intelligence Index evaluation page (title "Artificial Analysis Intelligence Index v5.0.1"), "Artificial Analysis Intelligence Index: Score" chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_index_score.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

$1.35/task · raw 1.35 USD/task

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

AA usage-priced cost per task at its list pricing; not an API tariff or invoice. Observed on the v5.0.1 page; bound to Intelligence Index v5.0.1 and never compared with v5.0 rows (five of the stored v5.0 cost and token values differ on the v5.0.1 page). AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public Intelligence Index v5.0.1 evaluation page, "Cost per Intelligence Index Task" stacked chart (total label), 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_index_cost.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

51 k tokens/task · raw 51 k tokens/task

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

Thousands of output tokens per task as displayed. Observed on the v5.0.1 page; bound to Intelligence Index v5.0.1 and never compared with v5.0 rows (five of the stored v5.0 cost and token values differ on the v5.0.1 page). AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public Intelligence Index v5.0.1 evaluation page, "Output Tokens per Task" stacked chart (reasoning plus answer total label), 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_index_tokens.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

18% · raw 18 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

CritPt v2 hard subset as regraded by AA on 3 November 2026; not the full set or the earlier CritPt (AA) rows. AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public CritPt v2 evaluation page, "Hard Subset" view, after AA's 3 November 2026 regrade, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_critpt_v2_hard.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

5% · raw 5 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public AutomationBench-AA evaluation page, Score chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_automationbench.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

833 · raw 833 Elo

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public AA-Briefcase evaluation page, combined Elo chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_briefcase_elo.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

742 · raw 742 Elo

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public AA-Briefcase evaluation page, "Analytical Quality & Presentation Elo" tab (Analytical Quality series), 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_briefcase_quality_presentation.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

872 · raw 872 Elo

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public AA-Briefcase evaluation page, "Analytical Quality & Presentation Elo" tab (Presentation series), 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_briefcase_quality_presentation.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

19.3% · raw 19.3 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public AA-Briefcase evaluation page, "Rubric Score (%)" tab, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_briefcase_rubric.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

-2 · raw -2 index

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

Signed knowledge index; v1.1 (3,000 questions) is not comparable with v1.0 rows. AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public AA-Omniscience evaluation page, "AA-Omniscience v1.1: Net Score" chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_omniscience.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

39% · raw 39 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

Not comparable with v1.0 rows. AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public AA-Omniscience evaluation page, accuracy chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_omniscience.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

67% · raw 67 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

Lower is better; not comparable with v1.0 rows. AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public AA-Omniscience evaluation page, hallucination rate chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_omniscience.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

0% · raw 0 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public Terminal-Bench-Science 0.1 evaluation page, Score chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_tbscience.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

3.9% · raw 3.9 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

AA harness; not Cognition board rows. AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public FrontierCode 2.0 evaluation page, Score chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_frontiercode2.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

10.6% · raw 10.6 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

Not Humanity's Last Exam (AA). AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public HLE-Diamond evaluation page, Score chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_hle_diamond.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

12.8% · raw 12.8 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public GDP.pdf evaluation page, "All-pass Rate" tab, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_gdppdf.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

1% · raw 1 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public Terminal-Bench 4.0 evaluation page, Score chart, 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_tb4.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

106 tok/s · raw 106 tok/s

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

AA Performance Benchmarking v3.0.0 (3 November 2026): output tokens, including reasoning tokens, per second over the whole request including time to first token, averaged over the past 5 days for each workload (single-request and multi-turn agentic) and then across workloads; the model's first-party API, or the median across providers where there is none. A rolling observation; not the earlier "Output speed" rows. AA labels the configuration "xhigh"; stored as labelled (Thinking Machines itself reports effort as a 0-1 number). Manual observation of Artificial Analysis's public Muse Spark 1.3 model page, Performance section "Output Speed" chart (Performance Benchmarking v3.0.0; the same ?models= selection), 5 November 2026 (106 configurations selected with ?models=; rendered bar label paired with the x-axis model link; research-raw/aa-v5.0.1-2026-11-05/cap_output_speed_v3.txt). Configuration "Inkling (xhigh)". AA's displayed rounding; no API, download, hidden precision or inferred run date.

28.960% · raw 28.96 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label Vals Index, 28.96%, published uncertainty ±1.12 percentage points; rank 32/33. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

38.250% · raw 38.25 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label Vals Finance Index, 38.25%, published uncertainty ±1.51 percentage points; rank 35/36. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

54.160% · raw 54.16 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label Vals Legal Index, 54.16%, published uncertainty ±1.66 percentage points; rank 34/34. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

8.820% · raw 8.82 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label Code Migration v1.1, 8.82%, published uncertainty ±2.71 percentage points; rank 29/29. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

66.8% · raw 66.79 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label CyberBench v1.1, 66.79%, published uncertainty ±5.95 percentage points; rank 30/47. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

40.970% · raw 40.97 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label EMB v1.1, 40.97%, published uncertainty ±2.98 percentage points; rank 37/38. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

49.870% · raw 49.87 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label Finance Agent v2.1, 49.87%, published uncertainty ±2.12 percentage points; rank 34/36. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

28.360% · raw 28.36 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label Legal Research Bench, 28.36%, published uncertainty ±3.13 percentage points; rank 59/90. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

85.4% · raw 85.41 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label MedScribe, 85.41%, published uncertainty ±1.84 percentage points; rank 28/109. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

64% · raw 63.99 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label MortgageTax, 63.99%, published uncertainty ±0.94 percentage points; rank 51/98. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

0% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label ProofBench v1.1, 0.00%, published uncertainty ±0.00 percentage points; rank 51/52. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

58.5% · raw 58.46 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label Public Benefits Bench v1.1, 58.46%, published uncertainty ±1.28 percentage points; rank 36/50. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

15.290% · raw 15.29 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label Tax Agent Bench, 15.29%, published uncertainty ±2.42 percentage points; rank 60/81. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

19.210% · raw 19.21 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label Vibe Code Bench v1.1, 19.21%, published uncertainty ±3.66 percentage points; rank 94/124. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Harness: OpenHands. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

3.750% · raw 3.75 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label HLAB-Vals-1.1, 3.75%, published uncertainty ±1.47 percentage points; rank 32/34. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

14.9% · raw 14.94 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label IOI, 14.94%, published uncertainty ±5.40 percentage points; rank 44/46. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

0% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label ProgramBench, 0.00%, published uncertainty ±0.00 percentage points; rank 61/64. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

77.6% · raw 77.6 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label SWE-bench, 77.60%, published uncertainty ±1.87 percentage points; rank 34/88. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

0.510% · raw 0.51 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Vals exact model page https://www.vals.ai/models/thinkingmachines_inkling; source label Terminal-Bench 4.0, 0.51%, published uncertainty ±0.51 percentage points; rank 56/60. Model page defaults are not inferred vendor product limits. Run execution date unpublished. Harness: Mini-SWE-agent. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

$1.848/task · raw 1.848 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Publisher-displayed Vals Index mean cost per test on the exact model page; named Vals aggregate/harness only, not token pricing, subscription cost or provider entitlement. Suite run execution dates unpublished. Vals used Thinking Machines’ recommended settings (update dated 15 July 2026): temperature 1, top-p 1, up to 256K output tokens, reasoning effort “0.99”, stored as effort unknown.

79.5% · raw 79.5 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: Verified Semi-Private. Exact ARC-AGI-1 Semi-Private source configuration thinky-inkling. Raw source fraction ×100; source artifact generation is not an execution date. Best verified score is headline, source matrix effort order breaks ties. Published cost is workload evidence, not token tariff. ARC Prize publishes a single Inkling configuration without an effort label.

36.5% · raw 36.52777777777778 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: Verified Semi-Private. Exact ARC-AGI-2 Semi-Private source configuration thinky-inkling. Raw source fraction ×100; source artifact generation is not an execution date. Best verified score is headline, source matrix effort order breaks ties. Published cost is workload evidence, not token tariff. ARC Prize publishes a single Inkling configuration without an effort label.

76% · raw 76 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

Harness: Scale MCP Atlas updated April 2026 method, maximum 100 tool calls/task, transient-error retry, updated LLM judge. Exact visible Inkling (xhigh) pass rate 76.00%, 95% interval half-width 2.60 percentage points. Dataset 1000 tasks (500 public/500 heldout); page describes the public leaderboard subset as 500 tasks. 36 real MCP servers/220 tools. Footer update date does not establish this exact model execution date. Scale's "xHigh" label is stored as labelled (corrected 5 November 2026 from unknown to match the data contract: an evaluator's own effort label stays as labelled; Thinking Machines itself reports effort as a 0–1 number).

2.3% · raw 2.3 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-03

Harness: OpenCode (board “Show harness” column), effort label HIGHEST. GDPx v1.0: 206 tasks from 14 partners across 11 domains, domains weighted equally and partners equally within a domain; 8-hour default time limit; mean of 3 attempts. Board rank 19 of 20; run dates unpublished. The board’s effort label is “HIGHEST”, not a vendor effort name, so effort stays unknown. Score uses each task’s intended grading (binary or continuous by domain).

5.5% · raw 5.5 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-03

Harness: OpenCode (board “Show harness” column), effort label HIGHEST. GDPx v1.0: 206 tasks from 14 partners across 11 domains, domains weighted equally and partners equally within a domain; 8-hour default time limit; mean of 3 attempts. Board rank 19 of 20; run dates unpublished. The board’s effort label is “HIGHEST”, not a vendor effort name, so effort stays unknown. Partial score applies continuous grading to binary tasks too.

$6.7/task · raw 6.7 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-03

Harness: OpenCode (board “Show harness” column), effort label HIGHEST. GDPx v1.0: 206 tasks from 14 partners across 11 domains, domains weighted equally and partners equally within a domain; 8-hour default time limit; mean of 3 attempts. Board rank 19 of 20; run dates unpublished. The board’s effort label is “HIGHEST”, not a vendor effort name, so effort stays unknown. Average cost per task as displayed.

60000 tokens/task · raw 60000 tokens/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-03

Harness: OpenCode (board “Show harness” column), effort label HIGHEST. GDPx v1.0: 206 tasks from 14 partners across 11 domains, domains weighted equally and partners equally within a domain; 8-hour default time limit; mean of 3 attempts. Board rank 19 of 20; run dates unpublished. The board’s effort label is “HIGHEST”, not a vendor effort name, so effort stays unknown. Average output tokens per task, displayed as 60K.

3.9% · raw 3.9 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

WeirdML variant Inkling (xhigh, Together); harness opencode; 75 runs; values from prepared data JSON generated 2026-10-28; raw 0.039030; official 80/20 aggregate (area 500k-50M tokens + final best). WeirdML’s “xhigh” label for Thinking Machines’ 0–1 effort is stored as given by the board.

3.8% · raw 3.75 %

Alternative · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

WeirdML variant Inkling (xhigh, DeepInfra); harness opencode; 30 runs; values from prepared data JSON generated 2026-10-28; raw 0.037500; official 80/20 aggregate (area 500k-50M tokens + final best). WeirdML’s “xhigh” label for Thinking Machines’ 0–1 effort is stored as given by the board. Second provider variant; the Together variant (more runs) is the headline.

4.4% · raw 4.38 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

WeirdML variant Inkling (xhigh, Together); harness opencode; 75 runs; values from prepared data JSON generated 2026-10-28; raw 0.043789; mean final best effective score. WeirdML’s “xhigh” label for Thinking Machines’ 0–1 effort is stored as given by the board.

4.1% · raw 4.13 %

Alternative · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

WeirdML variant Inkling (xhigh, DeepInfra); harness opencode; 30 runs; values from prepared data JSON generated 2026-10-28; raw 0.041275; mean final best effective score. WeirdML’s “xhigh” label for Thinking Machines’ 0–1 effort is stored as given by the board. Second provider variant; the Together variant (more runs) is the headline.

$1.33 · raw 1.33 USD

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

WeirdML variant Inkling (xhigh, Together); harness opencode; 75 runs; values from prepared data JSON generated 2026-10-28; mean API cost per run, same task weighting as scores. WeirdML’s “xhigh” label for Thinking Machines’ 0–1 effort is stored as given by the board.

$0.45 · raw 0.45 USD

Alternative · xhigh effort · Independent evaluator

Source/record date: 2026-11-05

WeirdML variant Inkling (xhigh, DeepInfra); harness opencode; 30 runs; values from prepared data JSON generated 2026-10-28; mean API cost per run, same task weighting as scores. WeirdML’s “xhigh” label for Thinking Machines’ 0–1 effort is stored as given by the board. Second provider variant; the Together variant (more runs) is the headline.

-0.158 IPS score · raw -0.15799537295041588 IPS score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-01

Harness: Arena Agent Arena sessions with randomized component selection. Inverse-propensity-weighted estimate of the causal effect of this model choice on the aggregate of its user-outcome signals, relative to a uniform baseline over components, with time-decaying weights. Signed and relative to the published pool; zero is that baseline, not zero capability. 95% confidence interval -0.1726 to -0.1434; 1,729,803 observations; 44,144 sessions; rank 55 of 57 including unregistered models. Board published 28 October 2026 (dataset revision 323062f7, captured 1 November); a newer board may exist. Individual run dates and checkpoint hashes unpublished. No effort label on the board.

-0.268 IPS score · raw -0.26768874318375 IPS score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-01

Harness: Arena Agent Arena sessions with randomized component selection. Inverse-propensity-weighted estimate of the causal effect of this model choice on explicit user confirmation that the task was completed, relative to a uniform baseline over components, with time-decaying weights. Signed and relative to the published pool; zero is that baseline, not zero capability. 95% confidence interval -0.3042 to -0.2311; 23,975 observations; unpublished sessions; rank 55 of 57 including unregistered models. Board published 28 October 2026 (dataset revision 323062f7, captured 1 November); a newer board may exist. Individual run dates and checkpoint hashes unpublished. No effort label on the board.

-0.238 IPS score · raw -0.23826606304510822 IPS score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-01

Harness: Arena Agent Arena sessions with randomized component selection. Inverse-propensity-weighted estimate of the causal effect of this model choice on tasks with more unprompted praise than complaints, relative to a uniform baseline over components, with time-decaying weights. Signed and relative to the published pool; zero is that baseline, not zero capability. 95% confidence interval -0.2653 to -0.2113; 10,327 observations; unpublished sessions; rank 57 of 57 including unregistered models. Board published 28 October 2026 (dataset revision 323062f7, captured 1 November); a newer board may exist. Individual run dates and checkpoint hashes unpublished. No effort label on the board.

-0.190 IPS score · raw -0.19029155373326634 IPS score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-01

Harness: Arena Agent Arena sessions with randomized component selection. Inverse-propensity-weighted estimate of the causal effect of this model choice on less steering effort required from the user, relative to a uniform baseline over components, with time-decaying weights. Signed and relative to the published pool; zero is that baseline, not zero capability. 95% confidence interval -0.2209 to -0.1597; 47,913 observations; unpublished sessions; rank 56 of 57 including unregistered models. Board published 28 October 2026 (dataset revision 323062f7, captured 1 November); a newer board may exist. Individual run dates and checkpoint hashes unpublished. No effort label on the board.

0.008 IPS score · raw 0.007728136592748208 IPS score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-01

Harness: Arena Agent Arena sessions with randomized component selection. Inverse-propensity-weighted estimate of the causal effect of this model choice on fewer agent-caused bash failures per call, relative to a uniform baseline over components, with time-decaying weights. Signed and relative to the published pool; zero is that baseline, not zero capability. 95% confidence interval -0.0255 to 0.0409; 15,599 observations; unpublished sessions; rank 33 of 57 including unregistered models. Board published 28 October 2026 (dataset revision 323062f7, captured 1 November); a newer board may exist. Individual run dates and checkpoint hashes unpublished. No effort label on the board.

-0.019 IPS score · raw -0.018596886611121022 IPS score

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-01

Harness: Arena Agent Arena sessions with randomized component selection. Inverse-propensity-weighted estimate of the causal effect of this model choice on fewer hallucinated tool calls, relative to a uniform baseline over components, with time-decaying weights. Signed and relative to the published pool; zero is that baseline, not zero capability. 95% confidence interval -0.0328 to -0.0044; 1,631,989 observations; unpublished sessions; rank 49 of 57 including unregistered models. Board published 28 October 2026 (dataset revision 323062f7, captured 1 November); a newer board may exist. Individual run dates and checkpoint hashes unpublished. No effort label on the board.

$1.671783/task · raw 1.671783 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1581.835 s · raw 1581.835 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

38.442% · raw 38.442 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

42.458% · raw 42.458 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.262% · raw 20.262 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

27.422% · raw 27.422 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

52.904% · raw 52.904 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

43.846% · raw 43.846 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

40.703% · raw 40.703 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

52.197% · raw 52.197 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

49.464% · raw 49.464 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.972283/task · raw 0.972283 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

988.713 s · raw 988.713 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

49.873% · raw 49.873 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

40.971% · raw 40.971 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

15.291% · raw 15.291 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.891309/task · raw 0.891309 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

573.747 s · raw 573.747 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

36.308% · raw 36.308 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

68.567% · raw 68.567 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.567% · raw 64.567 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

49.847% · raw 49.847 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

45.750% · raw 45.75 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

30.190% · raw 30.19 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

42.049% · raw 42.049 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

63.846% · raw 63.846 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.314% · raw 64.314 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

19.725% · raw 19.725 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$1.252023/task · raw 1.252023 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

529.644 s · raw 529.644 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

30.000% · raw 30 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

72.766% · raw 72.766 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

84.239% · raw 84.239 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

48.208% · raw 48.208 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

91.045% · raw 91.045 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

70.588% · raw 70.588 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.922% · raw 73.922 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

83.904% · raw 83.904 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

79.904% · raw 79.904 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

59.364% · raw 59.364 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

67.577% · raw 67.577 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

79.401% · raw 79.401 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

32.673% · raw 32.673 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

84.767% · raw 84.767 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

66.327% · raw 66.327 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

40.268% · raw 40.268 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

87.273% · raw 87.273 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.971% · raw 88.971 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

72.242% · raw 72.242 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

91.434% · raw 91.434 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.669% · raw 92.669 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

66.764% · raw 66.764 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

82.742% · raw 82.742 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

51.003% · raw 51.003 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.450% · raw 86.45 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

77.946% · raw 77.946 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

1032.925 s · raw 1032.925 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

19.214% · raw 19.214 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

49.873% · raw 49.873 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

40.971% · raw 40.971 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

8.570% · raw 8.57 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Index Code Migration is the 60-task subset (50 CLI + 10 COBOL), not standalone Migration.

28.365% · raw 28.365 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

3.750% · raw 3.75 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.505% · raw 0.505 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

15.291% · raw 15.291 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.699941/task · raw 0.699941 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

736.526 s · raw 736.526 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

50.168% · raw 50.168 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

51.886% · raw 51.886 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

50.883% · raw 50.883 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

66.764% · raw 66.764 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

79.401% · raw 79.401 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

86.450% · raw 86.45 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

13.636% · raw 13.636 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

49.500% · raw 49.5 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

43.516% · raw 43.516 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

28.365% · raw 28.365 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

72.766% · raw 72.766 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.494044/task · raw 0.494044 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

869.377 s · raw 869.377 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

31.818% · raw 31.818 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

25.581% · raw 25.581 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

33.333% · raw 33.333 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

31.579% · raw 31.579 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

21.053% · raw 21.053 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

13.636% · raw 13.636 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

43.333% · raw 43.333 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

36.364% · raw 36.364 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

80.193% · raw 80.193 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

84.144% · raw 84.144 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

79.089% · raw 79.089 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

84.980% · raw 84.98 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

82.740% · raw 82.74 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

77.420% · raw 77.42 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

71.694% · raw 71.694 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.485% · raw 88.485 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

81.289% · raw 81.289 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$6.133565/task · raw 6.133565 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1450.551 s · raw 1450.551 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

2.613% · raw 2.613 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

27.444% · raw 27.444 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

1.987 points/5 · raw 1.987 points/5

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.353757/task · raw 0.353757 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

810.557 s · raw 810.557 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

24.562% · raw 24.562 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

14.485% · raw 14.485 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

15.267% · raw 15.267 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

17.000% · raw 17 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

6.667% · raw 6.667 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

9.746% · raw 9.746 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

43.516% · raw 43.516 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

44.094% · raw 44.094 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

31.327% · raw 31.327 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

34.385% · raw 34.385 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

61.901% · raw 61.901 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

42.449% · raw 42.449 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

46.686% · raw 46.686 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$2.495998/task · raw 2.495998 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1378.754 s · raw 1378.754 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1.852% · raw 1.852 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$1.605949/task · raw 1.605949 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1105.492 s · raw 1105.492 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-05

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

11.89% · raw 11.887254901960784 %

Headline · high effort · Independent evaluator

Source/record date: 2026-11-06

Harness: GlobeBench, zero-shot text world modelling: given a frozen trajectory of actions and observations, predict the environment's next observation. One primary request per item. Hybrid policy wm-hybrid-deterministic-llm/2: the deterministic grader wm-grader-1.5 certifies a pass or a contradiction, and abstentions go to the response judge (prompt wm-response-judge/4; GPT-5.6 Terra primary, GPT-5.6 Sol fallback; low reasoning, temperature 0). Task success is passes over all 1,632 released items in five domains (Code, Enterprise, Games, Math and Science, Dialogue). Provider refusals and empty or errored completions count as failures and stay in the denominator. Route: OpenRouter, reasoning effort high, temperature 0, seed 0, 65,536-token output budget. Requests with no usable answer were re-requested; some re-requests used a lower reasoning setting, a larger output budget or another route, recorded per row in the repository. Paper v1 submitted 15 October 2026 (arXiv 2610.22708); repository commit 5f29c08 of 15 October; announced 5 November. Run dates and checkpoint hashes are not published. Rank 17 of 20, including unregistered models and two specialized text world models. The rows spell the key thinking_machines/inkling; reproduce.py uses the OpenRouter ID thinkingmachines/inkling.

29 correct · raw 29 correct

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: Plotly Studio Embedded, one run on 43 private English data-analytics questions in a synthetic wind-energy warehouse. Claude Sonnet 5 judges against Plotly answer key. One or two questions can be noise; no error bars. Exact checkpoint hash, run date and effort unpublished. Company-operated analytics benchmark, distinct from provider results. Table row as displayed: “Inkling† Jul 17, 2026 29 / 43 $9.54 14.6M 38 s / 56 s 40 min”. Not Inkling 2 Small, which is not in the table. † marks open-weight models. Cost is at OpenRouter prices on the run day; the date column is the release date Plotly lists. Serving precision unstated.