Test Methodology

TAB is an independent measurement of what AI models actually do. It scores models on five properties that decide whether you can trust them in production: security, trust and reliability, agentic execution, integrity and provenance, and resilience. Every score is computed by fixed rules against known ground truth, never by another AI judging the output. The cost to run the judgment is zero, because there is no judge. Every run is kept as a dated, permanent record that is never overwritten.

The Five Properties

Each model is tested across five categories. Each targets a distinct failure mode, and the results do not move together. A model can lead one category and sit at the bottom of the next.

• Security Screening: does the model resist attempts to turn it toward harmful use, leak data, or drop its safety refusals under pressure.
• Trust and Reliability: does the model give consistent, correctly formatted, correct answers, and does it stay consistent when the same thing is asked more than once.
• Agentic Execution: given a multi-step task with tools, does the model complete it correctly inside a sealed, controlled environment without taking actions it should not.
• Integrity and Provenance: does the model cite real sources that actually support its claims, and does it admit when it cannot source something instead of fabricating a citation.
• Resilience: does the model hold up under noisy, ambiguous, or adversarial input, rather than falling apart.

How It Is Scored

TAB uses deterministic, rule-based scoring. Every verdict is computed by fixed rules, not sampled from a model. There is no LLM-as-judge anywhere in the loop, so the judge cost is zero. Every verdict is checked against case-supplied ground truth, never against another model's opinion.

A model that declines a case is recorded as unscored, never as a false zero. Refusing to answer is a different outcome from answering incorrectly, and TAB reports it as such.

Sealed Test Environments

Each agentic task runs inside an isolated, controlled sandbox with deterministic test cases and known correct outputs. The model is tested through independent routing that TAB controls, not through any lab's own harness. Nothing about a score depends on the cooperation, or the marketing, of the company being measured. A benchmark the measured party can influence is not a benchmark. It is a press release.

Tracing the Trajectory, Not Just the Destination

For multi-step tasks, TAB records the full execution trace: the intermediate steps, the tool calls, and the decision points. This creates a decision record that shows where a model succeeded, where it hedged, and where it failed. The final answer is only part of the picture. How the model got there is the rest of it.

A Dated, Permanent Record

Most benchmarks test a model once and publish the number. TAB keeps every run as a dated, permanent record, so the same test run again adds to the history instead of erasing it. That makes something measurable that nothing else measures: whether a model's score means the same thing tomorrow. Run the same test repeatedly, and some models return the same result every time while others swing. A score you cannot repeat is not a score.