The Hidden Token Tax on AI Agents: Tool Bloat

Key takeaways

• Agents that inject their entire tool list each turn pay for it in input tokens, on every single call.
• Ratel, an open-source context-engineering layer, returns only the top-ranked tools per turn using BM25 keyword retrieval, with no embeddings and no vector database.
• Reported benchmark deltas at a 180-tool pool: a Sonnet-class model at about -82% input tokens and -68% cost; an Opus-class model at -72% to -81% tokens.
• On a local model at a 100-tool pool, reported accuracy jumped from 8% to 77%, because less context means less drift.
• Your savings depend on catalog size and model, so model the number before trusting the headline.

Where does the token tax come from?

Tool definitions are not free. Every tool you register (name, description, JSON schema) is text that enters the model's context window, and it enters on every turn, not just when the tool is used. Register 180 tools and the model reads all 180 descriptions to make one call. You pay input tokens for 178 tools it will never touch this turn.

That is a fixed tax on every request, and it scales with your catalog. The more capable you make your agent, the more you pay per turn just to remind it what it can do.

What do the benchmark numbers show?

Ratel sits between the agent and its catalog and resolves only the handful of tools relevant to the current turn (roughly two), using BM25 retrieval over a text projection of each tool. No embeddings, no vector database, no inference on the retrieval path. Reported results from the MetaTool agent benchmark, at a pool of 180 tools:

• Sonnet-class frontier model: input tokens down about 82%, cost down about 68%, with an 8 point accuracy dip the authors note is closing.
• Opus-class model: tokens down 72% to 81%, with accuracy roughly at parity or better depending on version.
• Open-source cloud model (glm-class): about +12 points accuracy and -85% input tokens.
• Local model (qwen-class) at a 100-tool pool: accuracy from 8% to 77%.

The pattern: the bigger the catalog, the bigger the win. Below about 30 tools, the authors say skip it, the pool fits in the prompt cleanly.

Why do cost and accuracy move together?

This is the counterintuitive part. Normally you trade cost for quality. Here, cutting context often improves both, because a shorter, more relevant tool list reduces the long-tail drift where a model picks the wrong tool from a crowded menu. Fewer distractors, better selection, and a smaller bill at the same time.

How big is the win for your stack?

The honest answer is: it depends. An 82% input-token reduction on a 180-tool pool does not transfer to a 20-tool agent. To size it, you need your own inputs: catalog size, average turns per session, model mix, and your provider's input-token price.

That is a modeling exercise, not a guess. In Calcaas you can plug in your token profile and a blended price per 1M input tokens, then compare 'full catalog' against 'retrieved subset' to see the monthly delta and what it does to your cost per active user. The benchmark tells you the shape of the win; the model tells you the size of yours.

Takeaway: tool bloat is a per-turn tax you are probably paying silently. Retrieval can cut it hard, but confirm the number against your own token profile before you bank it.