Examenos

Official, independent and pricing data

Model profilesv c917915

Levanto Sage 2

Explore Levanto Sage 2: exact decision configuration, published specifications, source-linked benchmarks and API tariffs when available.

← All decision models · Compare this decision system →

Published API tariff

$0.05 per 1M input tokens

Levanto Labs · checked 2026-11-08

Free and Developer plan rate. Paid plans lower input to $0.046 (Starter), $0.042 (Pro) and $0.038 (Growth) per 1M tokens, plus a monthly fee. A decision answer is free; reasoning tokens and extraction output cost $10 per 1M, and each grounding search $0.10.

No output-token charge does not mean free requests. These are provider tariffs, separate from benchmark workload costs.

51 recorded observations. Published results retain their source and setup; no Examenos evaluation was run. Missing results are not zero. How to read these comparisons →

Evaluator-published · etsabary System 1 Decision Benchmark

Levanto Sage 2 · Evaluator-published · etsabary System 1 Decision Benchmark
EvaluationResultEvidence
System 1 Decision Benchmark v4 · All-question accuracy70.932%
Setup & source

Correct answers over all 2,157 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · Macro-category accuracy73.252%
Setup & source

Equal-weight mean over the 43 categories.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · Added 140 AQuA questions44.286%
Setup & source

Accuracy on the 140 questions new in v4.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 140

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · Accuracy among usable responses70.932%
Setup & source

Correct answers over usable responses only; shown beside coverage.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · Usable-response coverage100.000%
Setup & source

Share of the 2,157 inputs with a usable response.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · Median client time227.7 ms
Setup & source

Pooled primary client time, network and failed requests included.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · P95 client time345.1 ms
Setup & source

Pooled primary client time, network and failed requests included.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F01 · Lookup and filtering95.349%
Setup & source

Lookup and filtering: 43 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 43

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F02 · Forward conditionals95.652%
Setup & source

Forward conditionals: 46 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 46

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F03 · Necessary conditions71.154%
Setup & source

Necessary conditions: 52 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 52

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F04 · Boolean scope90.741%
Setup & source

Boolean scope: 54 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 54

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F05 · Quantifiers and sets91.837%
Setup & source

Quantifiers and sets: 49 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 49

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F06 · Comparisons95.000%
Setup & source

Comparisons: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F07 · Scheduling71.053%
Setup & source

Scheduling: 76 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 76

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F08 · One-to-one assignments85.075%
Setup & source

One-to-one assignments: 67 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 67

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F09 · Exact truth counting49.296%
Setup & source

Exact truth counting: 71 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 71

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F10 · Truth-teller consistency59.459%
Setup & source

Truth-teller consistency: 74 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 74

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F11 · Sufficiency and inconsistency87.805%
Setup & source

Sufficiency and inconsistency: 41 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 41

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F12 · Object swaps63.462%
Setup & source

Object swaps: 52 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 52

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F13 · Sequential procedures37.500%
Setup & source

Sequential procedures: 48 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 48

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F14 · Temporal intervals88.095%
Setup & source

Temporal intervals: 42 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 42

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F15 · Spatial tracking77.551%
Setup & source

Spatial tracking: 49 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 49

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F16 · Reference binding72.000%
Setup & source

Reference binding: 50 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 50

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F17 · Beliefs and perspectives84.000%
Setup & source

Beliefs and perspectives: 50 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 50

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F18 · Causal intervention72.000%
Setup & source

Causal intervention: 50 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 50

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F19 · Exceptions and priorities93.103%
Setup & source

Exceptions and priorities: 58 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 58

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F20 · Planning90.698%
Setup & source

Planning: 43 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 43

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F21 · Directed reachability97.674%
Setup & source

Directed reachability: 43 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 43

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F22 · Preference ordering92.157%
Setup & source

Preference ordering: 51 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 51

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F23 · Rule counterexamples98.077%
Setup & source

Rule counterexamples: 52 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 52

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F24 · Authority and stale records93.023%
Setup & source

Authority and stale records: 43 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 43

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F25 · Fault diagnosis88.000%
Setup & source

Fault diagnosis: 50 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 50

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F26 · Probabilistic risk60.000%
Setup & source

Probabilistic risk: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F27 · Value of information60.000%
Setup & source

Value of information: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F28 · Robust decisions without a prior40.000%
Setup & source

Robust decisions without a prior: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F29 · Strategic decisions57.500%
Setup & source

Strategic decisions: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F30 · Concurrent-history consistency52.500%
Setup & source

Concurrent-history consistency: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F31 · Information-flow security72.500%
Setup & source

Information-flow security: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F32 · Numerical pattern recognition69.565%
Setup & source

Numerical pattern recognition: 23 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 23

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · F33 · Quantitative reasoning38.000%
Setup & source

Quantitative reasoning: 200 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 200

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · P01 · Understanding requests and routing75.000%
Setup & source

Understanding requests and routing: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · P02 · Recognizing multiple requested actions75.000%
Setup & source

Recognizing multiple requested actions: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · P03 · Tracking current conversational intention62.500%
Setup & source

Tracking current conversational intention: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · P04 · Selecting a tool or requesting clarification55.000%
Setup & source

Selecting a tool or requesting clarification: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · P05 · Checking authorization and scope57.500%
Setup & source

Checking authorization and scope: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · P06 · Making decisions with untrusted content87.500%
Setup & source

Making decisions with untrusted content: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · P07 · Applying a contextual content policy80.000%
Setup & source

Applying a contextual content policy: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · P08 · Extracting and normalizing facts37.500%
Setup & source

Extracting and normalizing facts: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · P09 · Checking claims against supplied evidence57.500%
Setup & source

Checking claims against supplied evidence: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · P10 · Assigning severity or priority from a rubric72.500%
Setup & source

Assigning severity or priority from a rubric: 40 questions.

Levanto Sage 2 · reasoning off · direct API; Hosted native; collected 2026-11-02 to 2026-11-02 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

Reasoning explicitly off. Two returned choices disagreed with the highest probability in the same response and were scored as received.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source
System 1 Decision Benchmark v4 · Tariff-based cost per 1,000 questions0.02167 USD/1k decisions
Setup & source

Estimate from a published tariff for calls without a reported charge, in USD per 1,000 questions; kept apart from reported charges.

Levanto Sage 2 · reasoning off · direct API; System 1 v4 ledger.

2157 calls at a published tariff out of 2,157.

Tariff-based estimate from the same ledger. Known run subtotal divided by 2,157 and multiplied by 1,000, not the bill for a separate 1,000-call run. A partial value is only the known subtotal: missing charges stay missing, so the true cost is higher by an unknown amount. Providers, routes, dates and access terms differ; these are not current tariffs. Free public access is not a price and has no row. Development benchmark; not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Result source · Benchmark source