Examenos

30.6 · raw 30.6 index

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “AA Index (v5.0)”, column “Inkling 2 Small”, value “30.6”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Index value, not a percentage. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

56.1% · raw 56.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “Vals Index (v2.2)”, column “Inkling 2 Small”, value “56.1%”. Thinking Machines states it uses Vals AI’s reported score for this benchmark; Vals’ own rows are stored separately under the Vals evaluator. Vals Index v2.2. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

25.4% · raw 25.4 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “FrontierCode v2.0”, column “Inkling 2 Small”, value “25.4%”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Artificial Analysis’s run of Cognition’s FrontierCode 2.0; Cognition’s own board has no Inkling 2 Small entry at the check. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

72.1% · raw 72.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “DeepSWE v1.1”, column “Inkling 2 Small”, value “72.1%”. Thinking Machines states it uses Datacurve’s reported score. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

5.7% · raw 5.7 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “Terminal Bench Science v0.1”, column “Inkling 2 Small”, value “5.7%”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

81.6% · raw 81.6 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “Vibe Code Bench v1.1”, column “Inkling 2 Small”, value “81.6%”. Thinking Machines states it uses Vals AI’s reported score for this benchmark; Vals’ own rows are stored separately under the Vals evaluator. Version 1.1. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

86.3% · raw 86.3 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “SWEBench Pro v2 (Hard)”, column “Inkling 2 Small”, value “86.3%”. Thinking Machines states it uses Scale AI’s reported score for this benchmark. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

70% · raw 70 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “BioMysteryBench”, column “Inkling 2 Small”, value “70.0%”. Thinking Machines states it uses Vals AI’s reported score for this benchmark; Vals’ own rows are stored separately under the Vals evaluator. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

85.4% · raw 85.4 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “MCP AtlasFull / Public”, column “Inkling 2 Small”, value “85.4/85.0%”. Thinking Machines states it uses Scale AI’s reported score for this benchmark. Full set (first value of “Full / Public”). Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

85% · raw 85 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “MCP AtlasFull / Public”, column “Inkling 2 Small”, value “85.4/85.0%”. Thinking Machines states it uses Scale AI’s reported score for this benchmark. Public split (second value of “Full / Public”). Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

1527 · raw 1527 Elo

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “AA-Briefcase v1.1”, column “Inkling 2 Small”, value “1527”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Elo-style rating. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

1630 · raw 1630 Elo

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “GDPVal-AA v2.1”, column “Inkling 2 Small”, value “1630”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Elo-style rating. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

70.5% · raw 70.5 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “JobBench”, column “Inkling 2 Small”, value “70.5%”. Harness unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

40.4% · raw 40.4 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “Tau 3 Banking v1.0.1”, column “Inkling 2 Small”, value “40.4%”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. τ³-bench Banking v1.0.1. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

21.7% · raw 21.7 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “Harvey LAB v1.1”, column “Inkling 2 Small”, value “21.7%”. Thinking Machines states it uses Vals AI’s reported score for this benchmark; Vals’ own rows are stored separately under the Vals evaluator. The post names Vals’ HLAB-Vals-1.1 as the source. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

46.6% · raw 46.6 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “Legal Research Bench”, column “Inkling 2 Small”, value “46.6%”. Thinking Machines states it uses Vals AI’s reported score for this benchmark; Vals’ own rows are stored separately under the Vals evaluator. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

64.7% · raw 64.7 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “PerceptionBench”, column “Inkling 2 Small”, value “64.7%”. Harness unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

83.8% · raw 83.8 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “ScreenSpot Prooriginal / with Python”, column “Inkling 2 Small”, value “83.8/91.5%”. Original setting without tools (first value of “original / with Python”). Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

68.6% · raw 68.6 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “BabyVisionoriginal / with Python”, column “Inkling 2 Small”, value “68.6/93.7%”. Original setting (first value of “original / with Python”). Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

93.7% · raw 93.7 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “BabyVisionoriginal / with Python”, column “Inkling 2 Small”, value “68.6/93.7%”. With a Python tool (second value of “original / with Python”). Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

25.2% · raw 25.2 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “GDP.pdf”, column “Inkling 2 Small”, value “25.2%”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

81.6% · raw 81.6 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “MMMU Pro (Standard 10)”, column “Inkling 2 Small”, value “81.6%”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Standard setting with 10 options. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

85.6% · raw 85.6 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “Charxiv RQoriginal / with Python”, column “Inkling 2 Small”, value “85.6/87.7%”. No tools (first value of “original / with Python”). Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

87.7% · raw 87.7 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “Charxiv RQoriginal / with Python”, column “Inkling 2 Small”, value “85.6/87.7%”. With a Python tool (second value). Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

36.7% · raw 36.7 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “Chartographyoriginal / with Python”, column “Inkling 2 Small”, value “36.7/80.4%”. No tools (first value of “original / with Python”). Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

80.4% · raw 80.4 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “Chartographyoriginal / with Python”, column “Inkling 2 Small”, value “36.7/80.4%”. With a Python tool (second value). Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

83.3% · raw 83.3 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “AA-LCR v1.1”, column “Inkling 2 Small”, value “83.3%”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

70.6% · raw 70.6 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “LongBench v2”, column “Inkling 2 Small”, value “70.6%”. Harness unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

16.1 · raw 16.1 index

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “AA-Omniscience v1.1score”, column “Inkling 2 Small”, value “16.1”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Omniscience v1.1 index (−100 to 100), not an accuracy. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

83.8% · raw 83.8 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “FORTRESS (adversarial)”, column “Inkling 2 Small”, value “83.8%”. Higher is better: the rate of refusing harmful requests. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

79.7% · raw 79.7 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “FORTRESS (benign)”, column “Inkling 2 Small”, value “79.7%”. Higher is better: the rate of still answering safe look-alike requests. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

97.1% · raw 97.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “StrongREJECT (none)”, column “Inkling 2 Small”, value “97.1%”. Higher is better; “none” jailbreak setting as labelled. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

13.1% · raw 13.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “Terminal Bench v4.0”, column “Inkling 2 Small”, value “13.1%”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. The post attributes Terminal-Bench v4.0 to Artificial Analysis (Vals’ own Terminal-Bench 4.0 run shows 12.626, a different value). Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

85.1% · raw 85.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “Terminal Bench 2.1Inkling 2 Small: Pi harness”, column “Inkling 2 Small”, value “85.1%”. Inkling 2 Small run in the Pi harness, as the row label states. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

81.3% · raw 81.3 %

Alternative · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “Terminal Bench v2.1 (Terminus)”, column “Inkling 2 Small”, value “81.3%”. Terminus harness; kept as an alternative to the Pi-harness row, which is the headline. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

38.8% · raw 38.8 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “SWE-Marathon v1.1”, column “Inkling 2 Small”, value “38.8%”. Harness unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

81.2% · raw 81.2 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “SWEBench Multilingual”, column “Inkling 2 Small”, value “81.2%”. Harness unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

82.2% · raw 82.2 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “SWEBench Verified”, column “Inkling 2 Small”, value “82.2%”. Harness unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

90.2% · raw 90.2 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “BrowseCompwith context management”, column “Inkling 2 Small”, value “90.2%”. With context management; harness unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

90.9% · raw 90.9 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “DeepSearchQA”, column “Inkling 2 Small”, value “90.9%”. Harness unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

44.1% · raw 44.1 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “HLE (text only)”, column “Inkling 2 Small”, value “44.1%”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Text-only. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

93% · raw 93 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “GPQA Diamond”, column “Inkling 2 Small”, value “93.0%”. Sampling and repeats unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

95.8% · raw 95.8 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “HMMT Feb 2026”, column “Inkling 2 Small”, value “95.8%”. Sampling and repeats unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

92.8% · raw 92.8 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “MathVision”, column “Inkling 2 Small”, value “92.8%”. Split unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

73.3% · raw 73.3 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (post and model card, 5 November 2026), benchmark table, row “AA-Omniscience v1.1non-hallucination rate”, column “Inkling 2 Small”, value “73.3%”. Thinking Machines states it uses Artificial Analysis’s reported score for this benchmark, for its own and other models; not an Artificial Analysis record captured by Examenos. Omniscience v1.1 non-hallucination rate (1 − hallucination rate). Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

45.6% · raw 45.6 %

Headline · xhigh effort · Own vendor

Source/record date: 2026-11-05

Thinking Machines’ Introducing Inkling 2 Small (model card, 5 November 2026), benchmark table, row “AutomationBench v1.0.6”, column “Inkling 2 Small”, value “45.6%”. Version 1.0.6 as labelled; harness unpublished. Thinking Machines runs its evaluations at effort 0.99, which the post labels “xhigh”, temperature 1.0 and a 1M-token maximum trajectory (256K for Inkling-Small); stored as xhigh because the vendor names it so. Execution date unpublished; the table calls itself an evaluation snapshot.

96.5% · raw 96.5 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-07

Harness: Verified Semi-Private. Exact ARC-AGI-1 Semi-Private source configuration thinky-inkling-2-small-xhigh (“Inkling 2 Small (XHigh)”). Raw source fraction ×100; source artifact generation is not an execution date. ARC Prize lists one configuration, labelled XHigh. Published cost is workload evidence, not token tariff.

81.7% · raw 81.6667 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-11-07

Harness: Verified Semi-Private. Exact ARC-AGI-2 Semi-Private source configuration thinky-inkling-2-small-xhigh (“Inkling 2 Small (XHigh)”). Raw source fraction ×100; source artifact generation is not an execution date. ARC Prize lists one configuration, labelled XHigh. Published cost is workload evidence, not token tariff.

31 · raw 31 index

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Manual observation of Artificial Analysis’s public Intelligence Index v5.0.1 evaluation page, “Artificial Analysis Intelligence Index: Score” chart, 7 November 2026 (selection inkling-2-small, inkling-small, inkling; research_notes/Model and benchmark news November 7/captures/aa-index-browser.json). Configuration “Inkling 2 Small” (slug inkling-2-small); AA’s label states no reasoning effort. The same reading shows Inkling Small 13 and Inkling (xhigh) 12, equal to the stored rows. AA’s 5 November changelog reports the same 31. AA’s displayed rounding; no API, download, hidden precision or inferred run date.

85.4% · raw 85.4 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: Scale MCP Atlas updated April 2026 method, maximum 100 tool calls/task, transient-error retry, updated LLM judge. Exact visible “Inkling Small 2” pass rate 85.40%, 95% interval half-width 2.20 percentage points, marked NEW, rank 9. The board label “Inkling Small 2” is Inkling 2 Small: Thinking Machines’ 5 November post cites Scale’s MCP Atlas full-set value 85.4 for Inkling 2 Small. No effort label shown. Footer update date does not establish execution date.

69.504% · raw 69.504 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.529753/task · raw 0.529753 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

833.925 s · raw 833.925 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

71.595% · raw 71.595 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

67.086% · raw 67.086 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

55.861% · raw 55.861 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

62.222% · raw 62.222 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.659% · raw 73.659 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

69.976% · raw 69.976 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.120% · raw 73.12 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

83.423% · raw 83.423 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

68.264% · raw 68.264 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

61.470% · raw 61.47 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.283293/task · raw 0.283293 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

514.765 s · raw 514.765 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

67.536% · raw 67.536 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

69.504% · raw 69.504 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

38.909% · raw 38.909 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

67.536% · raw 67.536 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.175784/task · raw 0.175784 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

296.329 s · raw 296.329 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

56.585% · raw 56.585 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

78.029% · raw 78.029 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

79.183% · raw 79.183 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

58.988% · raw 58.988 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

64.990% · raw 64.99 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

47.091% · raw 47.091 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

67.402% · raw 67.402 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.936% · raw 88.936 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

80.792% · raw 80.792 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

42.417% · raw 42.417 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

21.667% · raw 21.667 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$1.234911/task · raw 1.234911 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1290.908 s · raw 1290.908 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

40.000% · raw 40 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

30.000% · raw 30 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

10.000% · raw 10 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

30.000% · raw 30 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

50.000% · raw 50 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

30.000% · raw 30 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

30.000% · raw 30 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

30.000% · raw 30 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

40.000% · raw 40 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

30.000% · raw 30 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

20.000% · raw 20 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.717% · raw 93.717 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

95.471% · raw 95.471 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

94.803% · raw 94.803 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

95.522% · raw 95.522 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.215% · raw 92.215 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

94.706% · raw 94.706 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

96.404% · raw 96.404 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

96.463% · raw 96.463 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

96.820% · raw 96.82 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

96.587% · raw 96.587 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.509% · raw 92.509 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

98.185% · raw 98.185 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.652% · raw 92.652 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.003% · raw 93.003 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

94.799% · raw 94.799 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.364% · raw 92.364 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

91.176% · raw 91.176 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

96.619% · raw 96.619 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

95.618% · raw 95.618 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

95.677% · raw 95.677 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

94.752% · raw 94.752 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.387% · raw 93.387 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.144% · raw 93.144 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

96.183% · raw 96.183 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

71.044% · raw 71.044 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

56.081% · raw 56.081 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$3.509318/task · raw 3.509318 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

3034.529 s · raw 3034.529 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

81.546% · raw 81.546 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

67.536% · raw 67.536 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

69.504% · raw 69.504 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

39.457% · raw 39.457 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Index Code Migration is the 60-task subset (50 CLI + 10 COBOL), not standalone Migration.

46.635% · raw 46.635 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

21.667% · raw 21.667 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

12.626% · raw 12.626 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

38.909% · raw 38.909 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.889% · raw 73.889 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.555946/task · raw 0.555946 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

813.380 s · raw 813.38 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

70.622% · raw 70.622 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.546% · raw 73.546 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

74.604% · raw 74.604 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

94.752% · raw 94.752 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.509% · raw 92.509 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

96.183% · raw 96.183 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

22.727% · raw 22.727 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

53.943% · raw 53.943 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

65.752% · raw 65.752 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

46.635% · raw 46.635 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

93.717% · raw 93.717 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

46.635% · raw 46.635 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.288586/task · raw 0.288586 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

735.191 s · raw 735.191 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

60.606% · raw 60.606 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

46.512% · raw 46.512 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

51.852% · raw 51.852 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

39.474% · raw 39.474 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

36.842% · raw 36.842 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

22.727% · raw 22.727 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

70.000% · raw 70 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

27.273% · raw 27.273 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

87.872% · raw 87.872 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

92.674% · raw 92.674 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

91.526% · raw 91.526 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.404% · raw 88.404 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

88.662% · raw 88.662 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

84.532% · raw 84.532 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

79.482% · raw 79.482 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

95.589% · raw 95.589 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

81.821% · raw 81.821 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

38.909% · raw 38.909 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$0.144342/task · raw 0.144342 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

414.041 s · raw 414.041 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

41.860% · raw 41.86 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

52.478% · raw 52.478 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

46.947% · raw 46.947 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

50.222% · raw 50.222 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

22.667% · raw 22.667 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

18.000% · raw 18 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

65.752% · raw 65.752 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

61.101% · raw 61.101 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

68.164% · raw 68.164 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

72.395% · raw 72.395 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

73.880% · raw 73.88 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

63.331% · raw 63.331 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

57.658% · raw 57.658 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

12.626% · raw 12.626 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$8.218043/task · raw 8.218043 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

6062.689 s · raw 6062.689 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-10

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

18.519% · raw 18.519 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

16.667% · raw 16.667 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

21.212% · raw 21.212 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

3.704% · raw 3.704 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

0.000% · raw 0 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: published Vals component protocol. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

81.546% · raw 81.546 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update.

$1.901278/task · raw 1.901278 USD/task

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

1537.412 s · raw 1537.412 s

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-07

Harness: OpenHands. Exact Vals task/version and source model. Requested effort, provider and diagnostics are retained with this observation; evaluation date and checkpoint hash are unpublished. Source update date is not execution date. Quality standard error is not a confidence interval. A source revision does not by itself establish a model update. Workload diagnostic; quality uncertainty does not describe its uncertainty. Not a token tariff or controlled efficiency ranking.

33.3% · raw 33.3 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-11-09

Scale Labs / Elorian primary live HSS board, frozen 522-task suite (288 images, 234 silent videos). Main pass@1 averages three attempts and requires all rubric criteria. Board label "Inkling Small 2", company Thinking Machines, rank 11, no effort label (version empty), so effort is unknown; ±4.20 is the board's confidence half-width. Mapped to Thinking Machines' Inkling 2 Small, the only second-generation Small checkpoint. Exact inference endpoint, output ceiling and frame configuration are not published. Entry createdAt 2026-11-09T00:51:46Z is source metadata, not proof of evaluation date. Public-media contamination and judge dependence apply.