Tool design and context efficiency

Tool design and context efficiency (avoiding tool-sprawl)

When you connect a model to tools, the temptation is obvious: wrap every endpoint of every API as a tool and give them all to the model. It works with 3 or 4. But when you add several MCP servers and reach dozens of tools, the system starts to get worse, not better. This lesson teaches you why it happens and how to avoid it through design.

What you'll learn

• What tool-sprawl is and why too many tools degrade the model (the "context rot").
• How two concrete mechanisms cut context usage, with real numbers: the Tool Search Tool and Programmatic Tool Calling.
• A simple heuristic to decide what to use based on how many tools you have.
• Why running code safely needs a sandbox (we cover it in depth in the security lesson).

The problem: tool-sprawl and "context rot"

Every tool you give the model takes up space in its context window (the working memory it reasons with) in two ways. First, the definitions: the name, the description, and the parameter schema of each tool travel in the prompt, always, even if they're not used. Second, the intermediate results: if the model calls tool after tool, each raw response piles up in the context.

The effect is twofold and bad. The context gets inflated (more tokens = more cost and slower), and — worse — selecting the right tool degrades: with 50 similar descriptions, the model chooses wrong more often. That loss of quality when the context fills up with noise is called context rot. Naively wrapping every API as a tool is the number-one cause.

Mitigation 1 — Tool Search Tool (loading definitions on demand)

Instead of sending all 50 full definitions up front, you give the model a single small search tool and mark the rest with defer_loading: true. The heavy definitions don't travel until the model needs them: it searches for the right tool, and only then is its full definition loaded on demand. The saving is huge: in Anthropic's measurements, the tools context dropped from ~77,000 to ~8,700 tokens.

Mitigation 2 — Programmatic Tool Calling (having the model write a script)

Instead of the model requesting a tool, receiving the raw result, requesting another, and so on — filling the context with each intermediate step — the model writes a script that calls the tools inside a sandbox (isolated environment) and only returns the final result. The intermediate data stays in the sandbox, not in the context. Anthropic's measurement: one flow went from 150,000 to 2,000 tokens — a reduction of ~98.7%. (Separately, and as a different result, Anthropic reported ~37% fewer tokens on complex research tasks; don't confuse it with the 150k→2k jump.)

Mitigation 3 — Fewer tools, and more "high-level" ones

The third lever is about design, not mechanism: instead of 20 tools that mirror 20 endpoints, expose a few high-level tools that do complete tasks. Less surface area, less ambiguity, better selection. Often this is the simplest solution and the first one you should try.

The heuristic (rule of thumb)

• Fewer than ~10 tools → don't overcomplicate: give them all directly. The sprawl doesn't hurt yet.
• Many MCP servers / 50+ tools → turn on the Tool Search Tool with defer_loading.
• Flows with many steps and large intermediate results → Programmatic Tool Calling.
• Whenever you can → redesign toward fewer, higher-level tools.

Key moments in the video

• 3
  — Tool Search Tool. How a single search tool + defer_loading avoids loading all the definitions up front. It's the video explanation of the jump from ~77k to ~8.7k tokens.
• 5
  — Programmatic Tool Calling. The model writes a script that orchestrates the tools in a sandbox and only returns the final result — hence the collapse from 150k to 2k tokens.

Quick glossary

• Tool-sprawl: the explosion of tools (one per endpoint) that inflates the context and confuses the model.
• Context window: the model's working memory, measured in tokens; it's limited.
• Context rot: the drop in quality when the context fills up with definitions/results that get in the way.
• Tool Search Tool: a search tool that, with defer_loading: true, loads the heavy definitions on demand.
• Programmatic Tool Calling: the model writes code that calls tools in a sandbox and returns only the final result.
• Sandbox: an isolated, minimal-permission environment where that code runs safely.

To download

• 📄 Cheat sheet — Context efficiency (1-page PDF: the three techniques, the numbers, and the heuristic).
• 📝 Exercises — audit a set of tools and decide which technique to apply in each case.

Before moving on

If you keep one idea: more tools isn't more power — it's more noise. Design for the context: few high-level tools, search with deferred loading when there are many, and execution in a sandbox when the intermediate steps are heavy. The security of the sandbox and the server we already covered in Module 3; in the next lesson we put it in perspective with a debate about these design decisions.