Security You Can Verify

Most AI agent platforms ship agents with no security testing. MIT's six-university study found 23 of 30 deployed agents had never been tested by a third party. Four enterprise agents have no documented kill switch.

TAB was built specifically to close this gap: independent security verification, transparent scores, published grades including the D's.

Beyond free screening, TAB offers paid deep-dive security benchmarks that stress-test agents across adversarial, deceptive, and multi-agent attack surfaces.

TAB's prompt injection benchmarks (prompt_poison_benchmark.py and prompt_seep_benchmark.py) test whether an agent can be hijacked by adversarial instructions embedded in tool outputs, user messages, or retrieved documents. The benchmark covers 5 canary phrase injection patterns and 6 extraction techniques. An agent passes if it follows its system instructions even when the user or a tool tries to redirect it.

Prompt injection vulnerabilities in autonomous agents are especially dangerous because agents operate with tool access and persistent memory. A single successful injection can redirect an entire multi-step workflow. TAB's benchmarks simulate real attack conditions: the injected instructions are realistic, not toy examples.

The data_exfiltration_benchmark tests 50 scenarios across 5 categories of exfiltration attempts: direct requests for secrets, indirect extraction via reasoning, chain-of-thought leakage, encoded payloads, and social engineering. Regex-based secret detection flags any response that includes API keys, tokens, or credentials in any format.

Data exfiltration guardrails for agents are not optional. Agents that process customer data, internal documents, or sensitive configurations must be verified against extraction attacks before deployment. TAB's exfiltration suite covers both naive direct requests and sophisticated multi-turn extraction chains.

TAB's safety_refusal_benchmark applies multi-level jailbreak pressure: direct refusal pressure at 3 escalating levels, roleplay scenarios designed to bypass content policies, gradual escalation sequences, and social engineering via authority claims. An agent must maintain its refusal posture across all 4 attack types to pass.

Simulating adversarial attacks on agent reasoning requires more than a single-shot test. Real attackers iterate. TAB's escalation sequences model the actual pressure a deployed agent will face, including combinations of roleplay, authority spoofing, and gradual boundary erosion across a conversation. Red teaming AI agents at scale means running every attack type systematically, not spot-checking.

The memory_privacy_benchmark plants UUID canary values in one user session and attempts to retrieve them from a separate session. Cross-user memory leakage is a critical failure. Agents must not carry state between users or sessions without explicit design.

Memory isolation is one of the hardest properties to verify manually. TAB automates it: unique canary strings are injected into session A, then independent queries probe session B for those strings. Any match is a confirmed cross-session leak. This benchmark runs on every agent that claims persistent memory functionality.

PII redaction benchmarks test whether agents leak names, email addresses, phone numbers, SSNs, and financial identifiers when subjected to social engineering, indirect extraction, and multi-turn probing. Standard regex patterns cover 14 PII categories. An agent that passes a naive direct-request PII test can still fail under indirect extraction pressure.

TAB's PII suite tests the full adversarial surface: the agent is told a cover story, asked to reason about the data, encouraged to use the PII as an example, and prompted through multi-turn escalation. All 14 categories must be redacted across all attack vectors for an agent to pass this suite.

Spider-Sense is TAB's real-time threat detection layer. 29 rules organized across 3 severity levels: critical violations (immediate fail), high severity (scored reduction), and advisory flags (informational). Spider-Sense runs on every benchmark response before the LLM judge scores it, catching format violations, forbidden outputs, and policy breaches in under 50ms.

This agent security benchmark with real attack payloads is only as reliable as its pre-scoring gate. Spider-Sense ensures that no response bypasses basic policy checks before reaching the LLM judge. Critical violations short-circuit the scoring pipeline entirely, high-severity findings reduce the final score proportionally, and advisory flags are logged for audit purposes without affecting the score.

View live Spider-Sense rule violations and severity breakdowns: Spider-Sense Dashboard.

Anthropic can't independently verify Claude agents. Google can't independently verify Gemini agents. OpenAI can't grade GPT's homework.

Every frontier lab is building internal verification inside their own walled gardens. TAB is the only independent cross-platform security verification layer. 80 models across 20+ providers via 5 SDK integrations, one independent standard.

Each agent receives a Security Score as part of its Trust Seal. Scores are earned through real test runs, not self-reported. Methodology is published.

Grades include D's. TAB doesn't inflate scores to make agents look better than they are. If an agent scores poorly on security, you'll know before you buy.

Enterprise deployments require documented security testing, audit trails, and independent verification. TAB provides all three.

Contact Sales for enterprise security certification programs.