Examenos

Mean of the Jev Decision Index, Sarvam’s recreation of the Artificial Analysis Decision Index and Sarvam’s internal Indian-language benchmark.

Sarvam launch run, 3 November 2026 post.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: Not published

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Sarvam’s run of the Jev Decision Index over its 37 public benchmarks.

Sarvam launch run, 3 November 2026 post.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: Not published

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Sarvam’s own recreation of the Artificial Analysis Decision Index, with AA’s private knowledge test replaced by curated hard questions that include an “I don’t know” option. Not an Artificial Analysis result; never enters aa-decision-index-v1.

Sarvam launch run, 3 November 2026 post.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: Not published

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Sarvam’s internal Indian-language benchmark; tasks and scoring unpublished.

Sarvam launch run, 3 November 2026 post.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: Not published

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Median per-request response time over 1,000 requests sent one at a time on one NVIDIA B200, with each model on its developers’ own inference code. Jev 1.14 and Microsoft-Decision-1 are API-only: the post plots their publicly reported times (196 ms and 85 ms) and gives 431 ms and 533 ms measured from Sarvam’s cluster, a different setup, so neither has a row.

Sarvam launch run, 3 November 2026 post. Timed by Sarvam on one B200.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: 1000

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Share of answers given with at least 90% confidence that are wrong. English: 37 public Jev Decision Index benchmarks, 226,088 questions. Lower is better; compare with the expected rate.

Sarvam launch run, 3 November 2026 post. Answers given with at least 90% confidence.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: 226088

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Mistake rate the model’s own confidence implies on the same answers. English: 37 public Jev Decision Index benchmarks, 226,088 questions. A reference for the actual rate, not a quality score.

Sarvam launch run, 3 November 2026 post. Answers given with at least 90% confidence.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: 226088

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Share of answers given with at least 90% confidence that are wrong. Indian languages: the same benchmarks in 10 Indian languages and Hinglish, 20,108 questions. Lower is better; compare with the expected rate.

Sarvam launch run, 3 November 2026 post. Answers given with at least 90% confidence.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: 20108

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Mistake rate the model’s own confidence implies on the same answers. Indian languages: the same benchmarks in 10 Indian languages and Hinglish, 20,108 questions. A reference for the actual rate, not a quality score.

Sarvam launch run, 3 November 2026 post. Answers given with at least 90% confidence.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: 20108

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Share of questions whose answer changes across shuffled option orders. 2,000 questions run four times with options shuffled into four orders. Prism’s row uses its fixed internal option order, which the API applies. Lower is better.

Sarvam launch run, 3 November 2026 post. 2,000 questions run four times with options shuffled into four orders.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: 2000

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

The same test with Prism reading options in the order sent, without its fixed-order step. 2,000 questions run four times with options shuffled into four orders. Lower is better.

Sarvam launch run, 3 November 2026 post. 2,000 questions run four times with options shuffled into four orders.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: 2000

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Share of 2,000 questions whose answer changes when the identical request is sent four times. Lower is better.

Sarvam launch run, 3 November 2026 post. Same request re-sent four times.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: 2000

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Accuracy on whether each recorded run completed its task. Sarvam’s set of 700 recorded agent runs (1,770 questions): AgentRewardBench web-agent runs and τ-bench conversations for success and review, Who&When and TRAIL failed runs for the responsible agent. Labels come from human experts or the environment’s own check, never another model.

Sarvam launch run, 3 November 2026 post.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: Not published

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Accuracy on whether each recorded run needs a person to review it. Sarvam’s set of 700 recorded agent runs (1,770 questions): AgentRewardBench web-agent runs and τ-bench conversations for success and review, Who&When and TRAIL failed runs for the responsible agent. Labels come from human experts or the environment’s own check, never another model.

Sarvam launch run, 3 November 2026 post.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: Not published

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Accuracy on which agent caused each failed run. Sarvam’s set of 700 recorded agent runs (1,770 questions): AgentRewardBench web-agent runs and τ-bench conversations for success and review, Who&When and TRAIL failed runs for the responsible agent. Labels come from human experts or the environment’s own check, never another model.

Sarvam launch run, 3 November 2026 post.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: Not published

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Accuracy over all 1,770 questions, which also include repeated loops, side effects, the first wrong step and the error type. Sarvam’s set of 700 recorded agent runs (1,770 questions): AgentRewardBench web-agent runs and τ-bench conversations for success and review, Who&When and TRAIL failed runs for the responsible agent. Labels come from human experts or the environment’s own check, never another model.

Sarvam launch run, 3 November 2026 post.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: 1770

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Accuracy on Sarvam’s internal benchmark of 5,517 questions across nine tasks built from vulnerabilities published since July 2026: phishing artifacts, jailbreak prompts and MITRE attack-type classification. Sarvam says it will release the set later.

Sarvam launch run, 3 November 2026 post.

Printed in the vendor's own table.

Sarvam's own runs of 25 decision models in its launch post, each model on its developers' own inference code. Sarvam-measured response times are medians over 1,000 requests sent one at a time on one NVIDIA B200. Never compare with the Jev board, Artificial Analysis or other vendors' runs. Not ranked here.

Checked / reported: 2026-11-04 · Samples: 5517

Snapshot: 2026-11-03 · Revision: Read 2026-11-04 · Published observations; result reuse terms not specified

Correct answers over all 2,157 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Equal-weight mean over the 43 categories.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Accuracy on the 140 questions new in v4.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 140

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Correct answers over usable responses only; shown beside coverage.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Share of the 2,157 inputs with a usable response.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Pooled primary client time, network and failed requests included.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Pooled primary client time, network and failed requests included.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Lookup and filtering: 43 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 43

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Forward conditionals: 46 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 46

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Necessary conditions: 52 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 52

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Boolean scope: 54 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 54

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Quantifiers and sets: 49 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 49

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Comparisons: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Scheduling: 76 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 76

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

One-to-one assignments: 67 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 67

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Exact truth counting: 71 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 71

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Truth-teller consistency: 74 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 74

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Sufficiency and inconsistency: 41 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 41

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Object swaps: 52 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 52

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Sequential procedures: 48 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 48

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Temporal intervals: 42 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 42

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Spatial tracking: 49 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 49

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Reference binding: 50 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 50

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Beliefs and perspectives: 50 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 50

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Causal intervention: 50 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 50

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Exceptions and priorities: 58 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 58

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Planning: 43 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 43

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Directed reachability: 43 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 43

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Preference ordering: 51 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 51

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Rule counterexamples: 52 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 52

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Authority and stale records: 43 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 43

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Fault diagnosis: 50 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 50

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Probabilistic risk: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Value of information: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Robust decisions without a prior: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Strategic decisions: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Concurrent-history consistency: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Information-flow security: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Numerical pattern recognition: 23 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 23

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Quantitative reasoning: 200 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 200

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Understanding requests and routing: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Recognizing multiple requested actions: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Tracking current conversational intention: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Selecting a tool or requesting clarification: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Checking authorization and scope: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Making decisions with untrusted content: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Applying a contextual content policy: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Extracting and normalizing facts: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Checking claims against supplied evidence: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Assigning severity or priority from a rubric: 40 questions.

Sarvam Prism · direct API; Hosted native; collected 2026-11-05 to 2026-11-05 America/Toronto (1 days); one primary response per input, no scored retries. Identity: Exact provider-returned model identifier; hidden weights are not independently attested.

129 no-answer transient failures were retried before the selected answers; their latency is excluded.

Development benchmark, not an independently sampled test: 2,157 questions in 43 categories, the unchanged 2,017-question v3 bank plus 140 AQuA questions with source keys. The 23 numerical-pattern (MAIQ) and 200 quantitative (AQuA) questions are borrowed; the rest is predominantly project-developed with model-guided development, and Jev plus retained Microsoft and Drex feedback are development exposure. The AQuA additions were selected for misses by Fastino, Mammoth, GPT-6 Sol, D3 Ultra and Drex Max, which spreads but does not remove selection bias. Earlier primary responses are retained and newly added inputs were collected in later sessions, so one configuration can span several dates and routes. Strict accuracy counts an unusable response as not correct in the fixed denominator. Timing pools primary client times with the network included. The scoring key is private. A different number from v2 is a new bank, not a model change. Not ranked here.

Checked / reported: 2026-11-08 · Samples: 40

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)

Estimate from a published tariff for calls without a reported charge, in USD per 1,000 questions; kept apart from reported charges.

Sarvam Prism · direct API; System 1 v4 ledger.

2157 calls at a published tariff out of 2,157. Sarvam's INR 4 per million input tokens at its illustrative INR 90 per USD, excluding tax; not a billing receipt.

Tariff-based estimate from the same ledger. Known run subtotal divided by 2,157 and multiplied by 1,000, not the bill for a separate 1,000-call run. A partial value is only the known subtotal: missing charges stay missing, so the true cost is higher by an unknown amount. Providers, routes, dates and access terms differ; these are not current tariffs. Free public access is not a price and has no row. Development benchmark; not ranked here.

Checked / reported: 2026-11-08 · Samples: 2157

Snapshot: 2026-11-06 · Revision: 224ee46dc1c16a8ecf2c154a2144b4d2fdc45de1 · Published summary observations; upstream task/result terms retained (AQuA-RAT Apache-2.0; MAIQ declared MIT, original question rights not established)