Reference implementations to get you started. Each example shows a minimal agent structure compatible with TAB's upload flow.
A minimal Python agent that implements the respond function. TAB calls this function with each benchmark prompt.
"""
TAB Agent — Python Reference Implementation
=============================================
Implement the 'respond' function below. TAB will call it with each
benchmark prompt and use your return value as the agent's answer.
"""
import json
import re
def respond(prompt: str) -> str:
"""
Main entry point called by TAB for every benchmark prompt.
Args:
prompt: The benchmark question or task as a plain string.
Returns:
Your agent's answer as a plain string.
"""
prompt_lower = prompt.lower().strip()
# ---- Example: simple keyword routing ----
if "summarize" in prompt_lower or "summary" in prompt_lower:
return summarize(prompt)
if "extract" in prompt_lower:
return extract_entities(prompt)
# Default: echo-style fallback (replace with your real logic)
return f"Received prompt ({len(prompt)} chars). Implement your logic here."
# --------------- helper functions ---------------
def summarize(text: str) -> str:
"""Produce a short summary of the input text."""
sentences = re.split(r'(?<=[.!?])\s+', text)
# Return first two sentences as a naive summary
return " ".join(sentences[:2]) if sentences else text[:200]
def extract_entities(text: str) -> str:
"""Extract simple entities (numbers, emails) from text."""
numbers = re.findall(r'\b\d+\.?\d*\b', text)
emails = re.findall(r'[\w.+-]+@[\w-]+\.[\w.]+', text)
result = {"numbers": numbers[:10], "emails": emails[:5]}
return json.dumps(result, indent=2)
A JavaScript agent exporting a handler function. Works identically with TypeScript (.ts).
/**
* TAB Agent — JavaScript / TypeScript Reference Implementation
* =============================================================
* Export a 'handler' function. TAB calls it with each benchmark
* prompt and uses the return value as your agent's answer.
*/
/**
* @param {string} prompt - The benchmark question or task.
* @returns {string} Your agent's answer.
*/
function handler(prompt) {
const lower = prompt.toLowerCase().trim();
// ---- Example: keyword routing ----
if (lower.includes("translate")) {
return translate(prompt);
}
if (lower.includes("classify") || lower.includes("categorize")) {
return classify(prompt);
}
// Default fallback (replace with your real logic)
return `Received prompt (${prompt.length} chars). Implement your logic here.`;
}
// --------------- helper functions ---------------
function translate(text) {
// Placeholder: real agent would call a translation API
return "Translation placeholder — integrate your translation service here.";
}
function classify(text) {
const categories = ["Technical", "Business", "Creative", "Support", "General"];
// Naive keyword-based classification
if (/code|api|bug|deploy/i.test(text)) return "Technical";
if (/revenue|sales|market/i.test(text)) return "Business";
if (/write|story|design/i.test(text)) return "Creative";
if (/help|issue|problem/i.test(text)) return "Support";
return "General";
}
module.exports = { handler };
For agents with dependencies, upload a .zip archive (max 5 MB). Structure your zip like this:
my-agent/
├── main.py # or main.js / main.ts — entry point
├── requirements.txt # Python deps (pip install) — optional
├── package.json # Node deps (npm install) — optional
└── utils/
└── helpers.py # Additional modules
TAB looks for main.py (or main.js / main.ts) as the entry point and will install any declared dependencies before running benchmarks.