Examenos

70.854% · raw 70.854 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5158.766,"stderr":0.928,"cost_per_test":18.69955,"token_totals":{"input_tokens":45087854954,"output_tokens":654942865,"reasoning_tokens":510663474,"cache_read_tokens":43320518175,"cache_write_tokens":1766865965},"temperature":1.0,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"Anthropic","harness":null} Source default includes fallback-served component tasks; do not interpret as pure original-model execution.

70.177% · raw 70.177 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":5071.089,"stderr":1.004,"cost_per_test":28.122421,"token_totals":{"input_tokens":31961837560,"output_tokens":499754233,"reasoning_tokens":381319414,"cache_read_tokens":30671467913,"cache_write_tokens":1290041704},"temperature":1.0,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"Anthropic","harness":null} Source default includes fallback-served component tasks; do not interpret as pure original-model execution.

69.237% · raw 69.237 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2697.953,"stderr":1.09,"cost_per_test":13.372731,"token_totals":{"input_tokens":13360828360,"output_tokens":146167290,"reasoning_tokens":59887853,"cache_read_tokens":12071919973,"cache_write_tokens":null},"temperature":1.0,"top_p":null,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"provider":"Google","harness":null}

66.787% · raw 66.787 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":3558.574,"stderr":1.066,"cost_per_test":6.809627,"token_totals":{"input_tokens":11051421288,"output_tokens":96958718,"reasoning_tokens":50663639,"cache_read_tokens":10611361430,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

65.023% · raw 65.023 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2018.879,"stderr":1.098,"cost_per_test":21.277828,"token_totals":{"input_tokens":4924948729,"output_tokens":96766878,"reasoning_tokens":59928445,"cache_read_tokens":4484842803,"cache_write_tokens":null},"temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"OpenAI","harness":null}

64.877% · raw 64.877 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2123.575,"stderr":1.067,"cost_per_test":10.647504,"token_totals":{"input_tokens":18224226557,"output_tokens":97195877,"reasoning_tokens":52653929,"cache_read_tokens":17645554776,"cache_write_tokens":0},"temperature":1.0,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Meta","harness":null}

63.944% · raw 63.944 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2664.03,"stderr":1.072,"cost_per_test":3.614054,"token_totals":{"input_tokens":5486504345,"output_tokens":103203979,"reasoning_tokens":64580497,"cache_read_tokens":5078686427,"cache_write_tokens":406706672},"temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"OpenAI","harness":null}

61.257% · raw 61.257 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2394.709,"stderr":1.055,"cost_per_test":0.65747,"token_totals":{"input_tokens":27106524900,"output_tokens":324131672,"reasoning_tokens":251666265,"cache_read_tokens":26546819188,"cache_write_tokens":null},"temperature":1.0,"top_p":1.0,"max_output_tokens":384000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"DeepSeek","harness":null}

59.729% · raw 59.729 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":3864.087,"stderr":1.111,"cost_per_test":8.524625,"token_totals":{"input_tokens":15044468726,"output_tokens":216344074,"reasoning_tokens":null,"cache_read_tokens":null,"cache_write_tokens":null},"temperature":1.0,"top_p":0.95,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Fireworks AI","harness":null}

58.708% · raw 58.708 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2988.332,"stderr":1.282,"cost_per_test":3.173012,"token_totals":{"input_tokens":23071048400,"output_tokens":765852692,"reasoning_tokens":673025505,"cache_read_tokens":21612053476,"cache_write_tokens":1458667521},"temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"Anthropic","harness":null}

58.564% · raw 58.564 %

Headline · xhigh effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2013.837,"stderr":1.013,"cost_per_test":11.625468,"token_totals":{"input_tokens":7301646245,"output_tokens":112664299,"reasoning_tokens":65804651,"cache_read_tokens":6541333803,"cache_write_tokens":null},"temperature":0.7,"top_p":0.95,"max_output_tokens":null,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"provider":"SpaceXAI","harness":null}

57.813% · raw 57.813 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":4355.892,"stderr":1.24,"cost_per_test":0.434692,"token_totals":{"input_tokens":9589713675,"output_tokens":116021012,"reasoning_tokens":76814385,"cache_read_tokens":9118039253,"cache_write_tokens":null},"temperature":1.0,"top_p":0.95,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":null,"provider":"Xiaomi","harness":null}

55.951% · raw 55.951 %

Headline · high effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2158.482,"stderr":1.141,"cost_per_test":6.064799,"token_totals":{"input_tokens":15367814865,"output_tokens":117528442,"reasoning_tokens":68777600,"cache_read_tokens":14301116094,"cache_write_tokens":null},"temperature":1.0,"top_p":null,"max_output_tokens":65536,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"provider":"Google","harness":null}

54.826% · raw 54.826 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":4027.975,"stderr":1.114,"cost_per_test":0.221566,"token_totals":{"input_tokens":14242629902,"output_tokens":130885725,"reasoning_tokens":79837760,"cache_read_tokens":13658045739,"cache_write_tokens":null},"temperature":1.0,"top_p":0.95,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":null,"provider":"Xiaomi","harness":null}

53.781% · raw 53.781 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":2597.835,"stderr":1.067,"cost_per_test":3.866285,"token_totals":{"input_tokens":18180722777,"output_tokens":306409526,"reasoning_tokens":243120444,"cache_read_tokens":17662797762,"cache_write_tokens":0},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","provider":"MiniMax","harness":null}

53.209% · raw 53.209 %

Headline · unknown effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":3562.636,"stderr":1.13,"cost_per_test":1.446335,"token_totals":{"input_tokens":10732575937,"output_tokens":186491684,"reasoning_tokens":140958004,"cache_read_tokens":10205230838,"cache_write_tokens":0},"temperature":0.9,"top_p":1.0,"max_output_tokens":64000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":null,"provider":"Tencent","harness":null}

52.955% · raw 52.955 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":4496.011,"stderr":1.001,"cost_per_test":7.169465,"token_totals":{"input_tokens":8947298492,"output_tokens":88481723,"reasoning_tokens":52225164,"cache_read_tokens":8512028610,"cache_write_tokens":null},"temperature":1.0,"top_p":0.95,"max_output_tokens":262144,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Moonshot AI","harness":null}

52.322% · raw 52.322 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":1943.192,"stderr":1.117,"cost_per_test":0.450147,"token_totals":{"input_tokens":10052286779,"output_tokens":202172287,"reasoning_tokens":167716705,"cache_read_tokens":9525013908,"cache_write_tokens":525731464},"temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"OpenAI","harness":null}

50.861% · raw 50.861 %

Headline · max effort · Independent evaluator

Source/record date: 2026-10-20

Harness: publisher benchmark protocol. GDP-weighted aggregate with revised Finance v2.1, EMB v1.1, HLAB v1.1, Code Migration v1.1 and all-pass Tax. Old Index versions are not comparable. Source default includes provider-refusal fallback tasks for Opus/Sonnet; failure-controlled scores are separate. Published summary updated 19 October 2026; exact run dates and serving weight SHA unpublished. Requested effort from published reasoning_effort/compute_effort is not a common compute budget. Raw source diagnostics: {"latency":3830.634,"stderr":1.255,"cost_per_test":1.133938,"token_totals":{"input_tokens":31476965553,"output_tokens":178936231,"reasoning_tokens":null,"cache_read_tokens":30533831295,"cache_write_tokens":null},"temperature":1.0,"top_p":0.95,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"provider":"Fireworks AI","harness":null}