Context Engineering: Stopping Token Bloat and Mid-Session Amnesia
From Agentic Software Development · 10 articles · latest
You ask a coding agent to refactor an authentication handler. On turn one, the rules are crystal clear: reuse the shared HTTP client, pass explicit timeouts, and never touch existing database migrations. Forty turns later—after three full-file reads, two verbose pytest stack traces, and a sprawling grep dump—the agent quietly writes a raw requests.get() call with no timeout and edits a migration file you told it to leave alone.
Your context window did not hit its token ceiling. You still had 80,000 tokens of headroom left on a 200k-token model. What broke the session was context rot and mid-session amnesia—the two failure modes that turn long coding runs into expensive cleanup work.
Why Context Windows Degrade Before They Overflow
Large context windows create a dangerous illusion: if a model accepts 200k or 1M tokens, teams assume they can throw entire repositories, architecture handbooks, and test logs into the prompt without paying a quality penalty. In practice, attention is a finite budget with diminishing returns. As Anthropic's engineering team documented in Effective context engineering for AI agents, every extra token of noisy tool output or marginally relevant prose dilutes the model's ability to attend to load-bearing instructions sitting earlier in the conversation.
Two dynamics cause mid-session failures even when plenty of nominal capacity remains:
- Signal-to-noise dilution ("context rot"): Unfiltered directory trees, 1,500-line helper files, and 300-line compiler dumps push your original constraints into the mushy middle of the attention window. The model technically "has" the rule, yet still violates it under competing local context.
- Lossy compaction amnesia: When an agent harness auto-summarizes history at 75–85% utilization, it compresses dozens of tool calls into a short recap. Unless critical decisions and invariants live outside the chat scrollback, compaction drops exact flag names, file paths, or edge-case constraints—and the agent spends the next ten turns rediscovering mistakes it already fixed.
Context engineering treats the working window like L1 CPU cache, not an archive drive—a discipline formalized in token budgeting. Your goal is to supply the smallest set of high-signal tokens required for the current step—and keep everything else externalized until needed. Even when leveraging prefix caching as recommended in Google's Gemini long-context guide, keeping the dynamic working tail crisp directly determines reasoning accuracy.
Progressive Disclosure: Stop Front-Loading Everything
The fastest way to poison every session is front-loading knowledge "just in case" the agent might need it. Replace upfront dumping with progressive disclosure across both instructions and source code.
Keep AGENTS.md Lean and Defer Runbooks to Skills
Your repository root AGENTS.md (or CLAUDE.md) is injected on every single turn. Keep it under roughly 120 lines: exact build/test commands, 3–6 non-derivable architectural invariants, hard "never" boundaries, and a pointer map. As we covered in Writing Your First AGENTS.md, rules a linter can enforce deterministically do not belong in prose.
Multi-step operational checklists—release notes formatting, database seeding, blog publishing—belong in lazy-loaded SKILL.md directories. At boot, the agent only sees a two-line name and trigger description (~15 to 60 tokens per skill) and loads the full runbook only when a task matches it. That separation keeps routine bug-fix sessions completely free of unrelated workflow instructions (see Skills vs. MCP).
Locate First, Slice Second
The single biggest source of self-inflicted token bloat during coding is reading whole files when you only need one function. Train both your prompts and your harness around a strict two-step read pattern:
# Bad: dumps 1,400 lines (~12,000 tokens) into working memory
cat src/services/billing_engine.py
# Good: locate exact line numbers first, then slice 80 lines around the target
rg -n "def calculate_proration|class InvoiceBuilder" src/services/
sed -n '180,260p' src/services/billing_engine.py
When a codebase supports structural codebase navigation (Tree-sitter AST queries, SCIP symbol graphs, LSP goToDefinition, or find_references), prefer symbol-level lookups over raw text dumps. Returning a 40-line function body and its callers costs ~250 tokens instead of 12,000.
Tip
Audit your file-read habits: If an agent opens three 1,200-line files before making a five-line edit, it has already burned 30,000 tokens on code it will never touch. Enforce line-ranged reads (StartLine / EndLine) for any file over 200 lines.
Keep Tool Outputs and MCP Schemas Lean
Even disciplined file reading fails if your tool layer floods the window from both ends: bloated tool definitions at turn zero and bloated tool responses during execution.
On the input side, every eager Model Context Protocol (MCP) tool registered at session startup injects its full JSON Schema into the system prompt. Exposing 60 granular REST wrappers as separate eager tools silently burns 15,000+ tokens before the user types a word. Group related operations behind coarse, action-parameterized tools (for example, entries with action: "list" | "get" | "create" | "update") or use lazy-loaded MCP schema discovery so unused tools consume zero prompt budget.
On the output side, never pipe raw test suites or compiler spew straight into the model:
# Noisy: emits passing test dots, deprecation warnings, and full local frames
pytest tests/
# Lean: stops fast, hides passing noise, keeps tracebacks short
pytest -q --tb=short --maxfail=3 tests/test_billing.py
For linters and static analyzers, strip informational chatter and emit compact JSON or SARIF envelopes containing only file, line, ruleId, and message. Turning a 250-line stack dump into an 80-token structured error preserves reasoning headroom for the actual fix.
Isolate Noisy Exploration in Subagents and Disk State
Some tasks genuinely require reading thirty files—mapping an unfamiliar event pipeline, auditing call sites across modules, or hunting down a flaky integration test in logs. If your primary coding agent runs that hunt inside its main thread, those 50,000 tokens of discarded search results stay baked into its conversation history for the rest of the session.
The clean architectural fix is subagent context isolation. Spin up a short-lived research subagent with read-only tools, let it burn 40k–60k tokens exploring the codebase inside its own disposable window, and require it to return a structured 200–400 token brief: exact file paths, line ranges, symbol signatures, and observed constraints. Your primary implementer starts editing with a spotless context window (a pattern we also pair with worktree isolation in Parallel Agents Without Merge Hell).
For multi-hour refactors, pair subagent isolation with on-disk state externalization:
- Write the active plan, verified file paths, and remaining subtasks into a scratch
plan.mdfile on disk. - Update checkboxes as steps pass verification.
- After any
/compactevent or session handoff, re-readplan.mdfirst so the agent restores exact ground truth instead of relying on lossy conversation summaries.
How Wikantik Fits In: Externalized Project Memory via MCP
Once an engineering team grows beyond one repository—or accumulates dozens of Architecture Decision Records (ADRs), shared API contracts, and incident runbooks—local Markdown files hit a wall. Pasting 50,000 tokens of architecture docs into every repository's AGENTS.md guarantees stale copies, cross-repo drift, and immediate context rot.
This is where Wikantik acts as an externalized long-term memory layer for coding agents. Instead of hoarding prose inside every repo, ADRs, system specs, and service contracts live once in a versioned, searchable wiki and enter the agent's window strictly on demand through /knowledge-mcp (16 read-only tools, isolated from write mutations on /wikantik-admin-mcp as described in Dual MCP Architectures).
Wikantik structures context delivery around two distinct moments rather than one brute-force RAG call:
- Moment A — Session-Start Briefings (
get_briefing): Before the agent writes code, a deterministicPreInvocation/UserPromptSubmithook (callingGET /api/briefing?format=md) or an explicitget_briefingMCP call pulls a single token-budgeted Markdown briefing (default 6,000 tokens, configurable from 200 to 24,000). As documented in Tiered Agent Briefings and the Wikantik Agent Context Guide, Wikantik fills that budget in strict priority order: prompt-refined sections, pinned load-bearing pages (pins), domain cluster hubs and high-in-degree member pages (clusters), and shared cross-project contracts those pages cite—degrading overflow pages into lightweight title-and-summary pointers. - Moment C — Mid-Session Grounding (
assemble_bundle): When an unfamiliar question pops up mid-refactor,assemble_bundle(documented in the Wikantik RAG Context Bundle reference) runs hybrid Lucene BM25 plus dense-vector Reciprocal Rank Fusion over heading-faithful chunks. It returns de-duplicated, version-pinned sections (slug @ version) and acoverage.confidencetelemetry signal (strong,partial,weak, orunknown). Crucially, per Wikantik's ADR-0001, the server never synthesizes an answer—it hands raw, cited evidence directly to your coding model. - Token-Budgeted Orientation (
get_page_for_agent) and Escalation: If an agent already knows which spec it needs but does not want to swallow a 3,000-word page,get_page_for_agentreturns a compact structural projection: summary, frontmatter metadata, verification status, heading outline, and related pages. Whenassemble_bundlereportspartialorweakcoverage, the agent escalates toretrieve_contextfor candidate discovery andread_pagesto batch-fetch full Markdown for specific slugs.
Wiring a coding harness like Claude Code or Antigravity to that external memory plane takes one entry in .mcp.json:
{
"mcpServers": {
"wikantik-knowledge": {
"type": "http",
"url": "https://wiki.wikantik.com/knowledge-mcp",
"headers": {
"Authorization": "Bearer ${WIKANTIK_KNOWLEDGE_API_KEY}"
}
}
}
}
Note
Least-privilege memory access: Routine coding sessions only need the read-only /knowledge-mcp server. Keep /wikantik-admin-mcp separate so a coding agent fixing a bug cannot accidentally rewrite architectural records mid-flight.
A Practical Context Budget Checklist
Before your next multi-file coding session, audit your setup against these five guardrails:
| Context Layer | Target Budget / Scope | Rule of Thumb |
|---|---|---|
Root AGENTS.md | Under 120 lines | Build/test commands, hard invariants, and doc pointers only |
| Procedural Runbooks | Lazy SKILL.md headers | ~15–60 tokens at boot; full steps loaded only on match |
| Source Code & Logs | Line slices + filtered errors | Locate with rg -n, read 50–150 lines, strip passing test chatter |
| Broad Exploration | Isolated subagent window | Burn 50k search tokens in a subagent; return a 300-token brief |
| Architecture & ADRs | Externalized in Wikantik | Fetch via get_briefing at start and assemble_bundle mid-task |
Stop treating your agent's context window like a junk drawer you only clean after it overflows. Keep root instructions tight, isolate noisy exploration, and offload project memory to an indexed external plane—your agent will remember turn-one rules on turn fifty.
References & Further Reading
- Anthropic Engineering: Effective context engineering for AI agents
- Google AI Studio: Long context and context caching
- Model Context Protocol: MCP Specification
- Wikantik Documentation: Context Engineering for Coding Agents & Wikantik Agent Context Guide
Comments