Taming Token Bloat in Claude Code: Context Hygiene and Optimization for MCP Tools

Anyone who has spent an afternoon running autonomous coding agents like Claude Code, Cursor Agent, or OpenCode has encountered the phenomenon of context exhaustion. You start with an empty session, give the agent an ambitious refactoring task, and within 30 to 45 minutes find yourself with an exhausted context window, sluggish response latencies, and an uncomfortably steep API bill.

Worse than the raw monetary cost is agent amnesia: when the context reaches its ceiling, the client forcibly compacts the conversation history. Suddenly, the agent forgets the files it touched twenty minutes earlier, repeats code changes that already failed, and loses the strategic thread of the task.

The root cause of this failure mode is almost always the same: raw MCP tool output dumping. Here is an analysis of why this happens, how virtualization and local indexing solve it, and how to apply these techniques to your own agent workflows.

The math behind context exhaustion

Language model context windows look deceptively large on paper — 200,000 tokens sounds like hundreds of pages of text. But in an agentic coding loop, tool responses consume space exponentially faster than human typing.

Consider the payload sizes of standard engineering MCP tools:

• A single browser snapshot (e.g., via Playwright MCP or Puppeteer) averages 50 to 60 KB of raw accessibility tree and DOM metadata.
• Fetching 20 open GitHub issues with comments consumes 55 to 60 KB.
• Reading an application server access log or test execution report frequently exceeds 100 to 200 KB.

If your coding agent performs two web searches, inspects a test failure, and reads a repository’s issue list during its planning phase, it can easily burn 60% to 75% of its entire context capacity before authoring its first line of code.

Every subsequent turn in the conversation re-submits that entire accumulated history, causing token burn to accelerate until forced compaction occurs.

The virtualization pattern: Don’t dump, index

Rather than streaming massive payloads directly into the model’s active attention window, modern agent architectures introduce a context virtualization layer.

[Raw Tool Output (56 KB)] 
         │
         ▼
[Context Virtualization Layer]
         │
         ├──▶ Stored in Local SQLite (FTS5 full-text index)
         │
         └──▶ Compact Pointer + Summary (299 bytes) ──▶ [LLM Active Context]

Under this pattern:

1. When an MCP tool produces a large response (such as a database query result, log file dump, or DOM snapshot), the virtualization layer intercepts the payload.
2. The raw text is saved into a local sandbox database (typically an embedded SQLite database configured with FTS5 for full-text search).
3. The agent receives only an acknowledgment containing a high-level summary and query pointers (e.g. Indexed 5,000 log lines in access.log [id: req_982]. Found 42 error entries.).
4. When the agent needs specific details, it runs targeted queries against the local index rather than keeping thousands of raw lines resident in its primary prompt window.

Real-world token reductions

In benchmarked environments, this architecture achieves dramatic efficiency gains:

• Browser automation snapshots: Reduced from 56 KB to 299 bytes (a 99% reduction in tokens).
• Log file analysis: A 5,000-line server access log (20 KB) is reduced to a 222-byte index reference. The agent queries only the specific HTTP 500 error patterns it needs to analyze.
• Session longevity: Extends active agent operation from 30 minutes to over 3 hours of continuous work before compaction is triggered.

Preserving continuity across compactions

While reducing token volume is essential, the second major benefit of decoupling context from storage is session continuity.

When standard coding agents reach their token limits, they trigger generic conversation compaction: older user messages and assistant responses are summarized by an LLM. During this compression, fine-grained operational knowledge is routinely lost:

• Which alternative library versions were already tested and rejected.
• Specific edge cases uncovered during earlier test runs.
• Temporary file paths and sandbox environment variables.

With an external context indexing layer in place, the system uses event hooks to monitor every file edit, git commit, and subagent task. When compaction occurs, it constructs a priority-tiered state snapshot (typically under 2 KB) containing:

• Current task status and verified assumptions.
• Blacklist of attempted fixes that failed.
• Active file targets and recent diff summaries.

This structured snapshot is re-injected into the fresh conversation turn, acting as an immutable save checkpoint. If the agent attempted a fix that failed 25 minutes ago, it will not repeat the mistake even after the conversational context has been compressed.

Actionable context hygiene rules for developers

Whether using dedicated context virtualization tools or configuring standard MCP servers in Cursor and Claude Code, follow these operational rules:

1. Never dump full log files into chat: Instead of having your agent run cat application.log, instruct it to run filtered searches (grep -E 'ERROR|FATAL' application.log | tail -n 50).
2. Isolate web research in subagents: Delegate broad research tasks to a dedicated subagent. Have the subagent perform the web searches and report back only its distilled conclusions to the main thread.
3. Limit browser snapshot verbosity: Configure browser MCP tools to return structured text or accessibility nodes rather than raw HTML dumps.
4. Prune obsolete tool outputs: When a debugging cycle completes and a bug is fixed, clear or checkpoint the conversation rather than continuing to drag thousands of lines of diagnostic output into future feature work.

By treating the context window as a precious, high-cost working cache rather than an unbounded dumping ground, you improve model reasoning quality, eliminate agent amnesia, and keep your API expenses under control.