The Pulse

Artificial Analysis Releases Decision Index v2 With 45% of the Index Built on Private Datasets

Artificial Analysis released Decision Index v2 on November 8, 2026, increasing the share built on private datasets to 45% from 25%. The revision adds a Decision Score for correctness and calibrated confidence and gives AA-Paperclip a 30% we

AI.info Team · Written with AI · fact-checked · 2026-11-09

45% of Decision Index v2 is now built on private datasets

Artificial Analysis released Decision Index v2 on November 8, 2026. On the overall Index, Drex Max v1.1 from Nace AI and GLiDE v1.2 from Fastino Labs each score 73, and Mammoth 27B from Base Labs scores 72. Private datasets now make up 45% of the Index, up from 25% in v1. Its new Decision Score rewards a model for both correctness and calibrated confidence: each question scores from 0 to 100, with correct answers above 50 and wrong answers below 50. Across an evaluation, the score equals 50 × (1 + Accuracy − Brier score). The measure averages payoff across confidence thresholds and does not depend on choosing a single cutoff.

AA-Paperclip measures whether models answer all 15 questions correctly

The new AA-Paperclip benchmark accounts for 30% of the Index and uses 1,500 private documents, including resumes, receipts, invoices and AI and machine learning research papers. Each document has five decisions and ten facts to extract; the key metric measures the share of documents where all 15 questions are answered correctly. Claude Haiku 5.5 (Max) passes 92% of documents and GPT-6 Luna (Max) passes 86%, while Drex Max v1.1 and GLiDE v1.2 lead the decision models at 63% and 62%. For Drex Max v1.1, all five decisions are correct on 71% of documents, while all ten extractions are correct on 88%.

AA-Symmetry measures consistency across equivalent questions

AA-Symmetry carries 15% of the Index and tests 120 base questions across 440 variations, including reordered options, swapped labels and negated yes/no questions. Models from Baseten lead the benchmark, driven by a high abstention rate; Mammoth 27B scores 96 and Elephant 9B scores 93. Mammoth 27B selects the same answer across variations on 99% of questions, compared with 91% for Drex Max v1.1, 88% for GLiDE v1.2 and 67% for Clef.

Decision model costs range from $14 to $859 per 1M Tasks

Celeris-1.1 Decision at $14, Decider v1.2 at $68 and Drex Max v1.1 at $102 sit on the Decision Index versus Cost Pareto frontier; Clef costs $859 per 1M Tasks. GLiDE v1.2 and Decider Flash v1.2 evaluate each question in a separate pass and bill the input each time, so an AA-Paperclip task uses about 63k input tokens, compared with about 9k for Drex Max v1.1. GPT-6 Luna (Max) scores 76 and costs $1,530 per 1M Tasks, while Drex Max v1.1 scores 73 and costs $102; their times per task are 11.34 seconds and 275 milliseconds, respectively. Celeris-1.1 Decision records the lowest Time per Task at 56 milliseconds, followed by Elephant 9B at 70 milliseconds and Jev 1.14 at 92 milliseconds. Mammoth 27B scores within 4 points of GPT-6 Luna (Max) on the Decision Index and is 118x faster.