token-context-mcp 按需擷取程式碼，補足靜態程式碼庫封裝的函數實作盲點

Over the past few months, we've seen fantastic innovations in static repository packing tools like ContextCraft (contextcraft-ai), repomix, and gitingest. If your workflow is copy-pasting a repository snapshot into the web interfaces of ChatGPT or Claude for an architectural high-level review, static budget packing (using Tree-sitter + PageRank + a knapsack solver) is genuinely great.

However, when using autonomous multi-step coding agents (Claude Code, Cursor, Google Antigravity, OpenAI Codex), developers run into a fundamental architectural bottleneck: the "Function Body Blind Spot".

The Structural Dilemma: Pre-Packing vs. On-Demand Pulling

1. Static Packing freezes the context before the agent starts investigating. To compress a 500k-token repository into an 8k or 32k token snapshot, a static packer must prune almost every file down to signatures only (SIGS) or directory outlines (TREE).
2. You cannot fix a bug inside a function body that was pruned down to a one-line signature. In real debugging, an agent reads Function A, realizes the root cause is inside callee Function B, and needs Function B's 100% complete implementation body plus its surrounding callers/callees.

Instead of pre-packing the whole repository upfront, we built token-context-mcp (v0.3.5) — a 100% local, zero-cloud-API Model Context Protocol (MCP) server that lets your coding agent progressively pull exact AST code context on demand.

How Progressive MCP Pull Works (The 5-Step Zoom-In)

Instead of cat-ing entire 5,000-line files or relying on a static pre-computed snapshot:

1. Global Orientation (get_repo_map): Builds a PageRank-weighted symbol map of the entire repository in ~682–703 tokens (tiktoken o200k_base) vs ~507k to 539k tokens of raw source (~0.13% of repo size).
2. Hybrid Localization (search_source + find_symbols): Combines SQLite FTS5 keyword search inside function bodies with AST symbol globbing.
3. Structural Skeletons (get_file_skeleton): Uses Tree-sitter byte spans to elide function bodies into # ... while preserving 100% of imports, class hierarchies, and type signatures (saves ~79%–82% tokens).
4. On-Demand Full Body Pull (inspect_symbol / get_symbol_context): Pulls the 100% complete source body of the exact target symbol + docstrings + 1-hop caller/callee signatures in a single call (~500–850 tokens, saving ~91.5% vs reading full files).
5. Blast-Radius Analysis (get_impact_slice / get_module_dependents): Traces multi-hop callers/callees and module imports before the agent edits a shared interface.

Empirical Benchmarks: 59-Task Head-to-Head & M10 Suite

Every token count below is verified with OpenAI's real tiktoken (o200k_base) tokenizer across two real-world codebases: rich (Python, 213 files, 507k tokens) and fastify (JS/TS, 291 files, 538k tokens). All scripts and raw JSON outputs are public in the repository.

Experiment 1: Does the target function's FULL BODY survive into context? (59 Tasks)

A task counts as a Full Body Hit if >=90% of the ground-truth target function's non-empty lines appear in the delivered context:

Key Takeaways:

• At a 32k budget with --focus, ContextCraft includes at least the signature for 50/59 (84.7%) of tasks — proving its PageRank + BM25F ranker is genuinely effective at building a high-level map.
• But when an agent needs the actual implementation body to write a code patch, on-demand MCP pull delivers 33/59 (55.9%) full bodies at ~7.6k tokens vs 2/59 (3.4%) for equal-budget static packing (16.5x more full bodies).
• For the 13/59 (22%) tasks where the zero-LLM deterministic 2-step pull missed, our standardized prompt protocol instructs the agent to fall back gracefully to direct file reading.

Experiment 2: Equal-Budget Retrieval vs. Truncated Grep (M10 Suite)

On 30 pre-registered tasks in rich at an identical ~1,900-token budget:

• File Recall: 0.90 (token-context-mcp) vs 0.57 (budget-truncated grep) — +58% improvement
• Symbol Recall: 0.54 vs 0.10 — 5.4x improvement

7-Dimension Technical Comparison Matrix

• 10 Core Read-Only Tools (Zero Schema Bloat): Exposes only 10 read-only retrieval tools by default to keep initial context small. Optional extensions unlock multi-agent SQLite memory with mutex locks, tool discovery, and local Ollama summarization.
• 3-Layer Governance Microkernel: Enforces clean AST boundary caps, per-tool execution timeouts (10s/5s), and a 150,000 cumulative token circuit breaker to prevent runaway billing loops.
• Local & Private: No arbitrary shell command execution, zero external telemetry, with automatic secret redaction ([REDACTED]).

Getting Started in 10 Seconds

token-context-mcp is open-source (MIT), indexed on the official Model Context Protocol Registry, and live on PyPI.

Run it directly without installation via uvx:

uvx token-context-mcp quickstart .

Or connect it to Claude Code:

claude mcp add --transport stdio --scope user token-context -- uvx token-context-mcp serve

Or add to Cursor Settings > Features > MCP Servers:

• Name: token-context
• Type: command
• Command: uvx token-context-mcp serve

• GitHub Repository: https://github.com/MCK564/token-context-mcp
• PyPI Package: https://pypi.org/project/token-context-mcp/
• Official MCP Registry: Registered as io.github.MCK564/token-context-mcp

Huge respect to the author of ContextCraft for advancing open-source context compression — static packers and interactive MCP servers solve complementary parts of the AI developer workflow!