Independent Measurement • 9 Models • 5 Categories • One Identical Case Set

Five Findings from Independent AI Model Measurement

The frontier premium won nothing, and the rankings invert by property. Nine models, five property categories, one identical set of cases. The most expensive model tested led no category outright, no model leads everywhere, and on the properties that matter most the scores move from one run to the next.

The measurement

One representative run per model on each of the five TAB Index categories. Scores are raw checks passed out of the checks available in each category. Category leaders are marked; ties are marked on every model that reached the top score.

Model Class Security /12 Trust /14 Agentic /15 Provenance /15 Resilience /15 Cost/run
claude-haiku-4-5 closed · low-cost 11 13 15 14 15 $0.0028
deepseek-v3.2 open-weight · lowest-cost 10 14 15 12 13 $0.0005
gpt-4.1-nano closed · low-cost 11 14 13 12 14 $0.0002
glm-5.2 open-weight · low-cost 10 14 15 15 15 $0.0110
claude-fable-5 frontier · Mythos-class refused 8/12 (unscored) 14 15 11 14 $0.0567
grok-4.5 closed · mid-cost 10 14 15 15 15 $0.0087
gpt-5.6-sol frontier · reasoning 9 14 15 12 14 $0.0166
gpt-5.6-terra frontier · reasoning 9 14 15 14 13 $0.0089
gpt-5.6-luna frontier · reasoning 12 14 15 13 14 $0.0061

◆ marks the leading score in a category; where models tie for the lead, each is marked. claude-fable-5's Security cell is a refusal: its safety classifier declined 8 of 12 cases, so the category is recorded as unscored, not as a zero the model never produced.

Five findings

Each finding reads directly off the run above. The numbers are measured, not modeled.

  1. The frontier premium won nothing.

    The Mythos-class model, at more than 300x the cheapest model's per-run cost, led no category outright. Its single best result is a shared tie on Agentic that several cheaper models also reached.

  2. Rankings invert by property.

    DeepSeek ties for the top on Trust and Agentic yet sits near the bottom on Provenance. GLM 5.2 (open-weight, roughly one fifth the priciest model's cost) ties for the lead on four of five categories and is held back only by Security. No single model leads everywhere, and no model sits at the bottom everywhere. A composite "best model" number averages that inversion away and hands the buyer the wrong answer for the property their business runs on.

  3. The most expensive model was the worst at telling you where its facts came from.

    On Integrity and Provenance, Anthropic's Claude Fable 5, the priciest model we test at roughly $0.057 per run, scored lowest of all nine models. It was beaten on provenance by models a fraction of its cost, including gpt-4.1-nano at $0.0002 per run, roughly 280 times cheaper. Two models scored a flawless 15 out of 15 in this category, and neither was the most expensive. Provenance decides whether a model cites real, supporting sources instead of inventing them, the property that matters most for legal, financial, and medical work, and here price bought the opposite of what you would expect.

  4. A refusal is not a zero.

    The frontier model's safety classifier declined most Security cases. TAB records that as unscored and reports it plainly, rather than assigning a number the model never produced.

  5. One run is not a verdict.

    Model outputs vary run to run, so a single score is a snapshot, not a verdict. Because every run is dated and kept rather than overwritten, that variance is itself visible in the record.

How this was measured

The measurement is reproducible and the scoring is inspectable. Nothing here depends on a model grading another model.

// methodology

  • Deterministic, rule-based scoring. Verdicts are computed by fixed rules, not sampled from a model.
  • No LLM judge. There is no model-as-judge in the loop, so the judge cost is $0.00.
  • Ground-truth only. Every verdict is checked against case-supplied ground truth, never against opinion.
  • Refusals recorded as unscored. A declined case is reported as unscored, never as a false zero.
  • Every run is a dated, immutable observation. Runs are kept, never overwritten, so change over time stays visible in the record.

Frequently Asked Questions

Does a higher-priced or frontier model score better?
No. In this measurement the most expensive model tested, at more than 300x the cheapest model's per-run cost, led no category outright. Its single best result was a shared tie on Agentic that several cheaper models also reached. Price did not predict measured quality on any of the five categories.
Why not publish one overall score?
Because the rankings invert by property. A model that ties for the bottom on Security can tie for the top on Trust and Agentic. A composite number averages that inversion away and hands the buyer the wrong answer for the property their business actually runs on. We publish each category separately so you can read the one that matters to you.
How is this different from a leaderboard?
Scoring is deterministic and rule-based, with no LLM judge, so the judge cost is zero and there is no model grading another model. Every verdict is checked only against case-supplied ground truth. And every run is a dated, immutable observation that is never overwritten, so the record shows change over time rather than a single number that quietly replaces the last one.
What does a refusal recorded as "unscored" mean?
When a model's safety classifier declines a case, the model never produced an answer to score. TAB records that cell as unscored and reports it plainly, rather than assigning a zero the model never earned. A refusal is not a failing score; it is the absence of a score, and it is labeled as such.
Is a single run enough?
No. Model outputs vary from one run to the next, so a single score is a snapshot, not a verdict. Because every run is dated and kept rather than overwritten, that variance is visible in the record instead of hidden behind one headline number.