多步驟編碼智能體如何透過「存檔式上下文壓縮」解決規格偏移，輸入 token 減少 91.9%

💡 AI Labo Benchmark Series — Experiment E12 (Ranked #6 out of 41 Patterns)

This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.

• 🚀 View All 41 Benchmark Results & Master Ranking
• 🛡️ Read How Our Proxmox & systemd Anti-Cheating Sandbox Works

Metadata & Experiment Identity

• Experiment ID: E12 (Backlog Candidate C008)
• Run ID: e12-20261028-vm216-01
• Sandbox Environment: Proxmox Isolated VM VM216 (ai-e12-216, 0 NICs, virtio-serial QGA relay, systemd UID separation)
• Model: gemini-3.8-flash (thinkingLevel: LOW)
• Pattern Under Test: Save-Game Context Compaction (/compact at Milestones) — "Unbounded History Accumulation vs. Milestone JSON State Compaction"

1. Executive Summary (TL;DR)

When an AI coding agent tackles a multi-step engineering task across several modules, keeping every prior file dump, telemetry log, and intermediate code output inside a single growing messages[] array (Unbounded Context Accumulation) causes two compounding problems:

1. Quadratic Token Inflation: Every past turn's tool output is re-billed on every subsequent turn (6,556 ➔ 11,769 ➔ 17,544 input tokens).
2. Mid-Task Spec Drift (Lost-in-the-Middle): By Turn 2 or Turn 3, fine-grained string normalization rules and boundary conditions buried in the middle of the history get drowned out by verbose telemetry logs—causing subtle bugs that break end-to-end integration tests.

To test the solution—Milestone Context Compaction (the "Save-Game" Pattern)—we benchmarked a 3-milestone power-grid telemetry and carbon-credit clearing pipeline (Milestone 1: collector.py ➔ Milestone 2: clearing.py ➔ Milestone 3: dispatch.py) inside isolated Proxmox VM VM216:

• Mode A (AccumulatedHistoryRunner): Appends all 3 milestone specifications, code outputs, and verbose telemetry audit dumps (VERBOSE_TELEMETRY_AUDIT_DUMP) into one continuous chat thread (0 compactions).
• Mode B (MilestoneCompactionRunner): Immediately after each milestone passes its deterministic verification gate, executes a /compact reset—discarding raw logs and replacing the chat history with a 120-token structured JSON checkpoint (SAVE_GAME_COMPACTION_V1) plus only the next milestone's specification.

Key Result

Mode A's accumulated history (35,869 cumulative input tokens) suffered a classic Lost-in-the-Middle failure on Milestone 2 (clearing.py), forgetting to apply .strip().upper() to contract_tier and failing 2 out of 20 hidden E2E tests (18 / 20, 90.0%). Mode B's Save-Game Milestone Compaction kept every turn under 1,400 input tokens (2,922 cumulative input tokens, -91.9% reduction), cut API cost by -72.2% ($0.0340 ➔ $0.0095), and achieved a flawless 20 / 20 (100%) pass.

2. Experiment Architecture: Unbounded History vs. Save-Game Compaction

+-------------------------------------------------------------------------+
| Proxmox Isolated VM216 (0 NICs, virtio-serial QGA, systemd UID split)   |
|  Target Task: 3-Milestone Power Grid & Carbon Clearing Pipeline         |
|               (M1: collector.py --> M2: clearing.py --> M3: dispatch.py)|
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: AccumulatedHistoryRunner (No Compaction)                  |  |
|  |  - Turn 1 (M1): 6,556 in  --> collector.py PASS                   |  |
|  |  - Turn 2 (M2): 11,769 in --> Audit logs drown out M2 spec!       |  |
|  |                               Forgets contract_tier.strip().upper()|  |
|  |  - Turn 3 (M3): 17,544 in --> Cumulative 35,869 input tokens      |  |
|  |  --> Final Score: 18 / 20 (Crashes on lowercase/padded tiers!)    |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: MilestoneCompactionRunner (Save-Game JSON Compaction)     |  |
|  |  - Turn 1 (M1): 650 in   --> collector.py PASS                    |  |
|  |  - [Compaction #1]: Discards raw logs; writes 120-tok JSON state  |  |
|  |  - Turn 2 (M2): 1,388 in --> Fresh context nails all M2 rules!    |  |
|  |  - [Compaction #2]: Updates verified_contracts in JSON checkpoint |  |
|  |  - Turn 3 (M3): 884 in   --> Cumulative 2,922 input tokens (-91.9%)| |
|  |  --> Final Score: 20 / 20 (100% Perfect Pass)                     |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+

3. Code-Level Postmortem: How Unbounded History Caused a Lost-in-the-Middle Bug

3.1 Mode A (AccumulatedHistory): The Missing .strip().upper() on Turn 2

Inspecting accum-bundle.json (Mode A's final submitted code) reveals a textbook example of Attention Dilution:

• On Turn 1 (collector.py), Mode A properly implemented .strip().upper() normalization for station_id and grid_zone.
• By Turn 2 (clearing.py), however, 45 lines of VERBOSE_TELEMETRY_AUDIT_DUMP_M1 plus the full Turn 1 code had bloated the prompt to 11,769 tokens. Buried in the middle of the prompt was the requirement that clearing.py must also normalize contract_tier via .strip().upper().
• Mode A completely missed that normalization rule in clearing.py:

# ❌ Mode A (accum-bundle.json — clearing.py generated at 11,769 input tokens)
def settle_grid(readings, contract_tier="BASE", credit_offset=0):
    if not isinstance(readings, list) or len(readings) == 0:
        raise ValueError("readings must be a non-empty list")
    # BUG: Forgot isinstance(contract_tier, str) and contract_tier.strip().upper()!
    if contract_tier not in ALLOWED_TIERS:
        raise ValueError(f"contract_tier must be one of {ALLOWED_TIERS}")

Because of that single omitted .strip().upper(), Mode A crashed with a ValueError on two hidden E2E integration tests that passed padded/lowercase tier strings (contract_tier=" green " in e2e_01 and contract_tier="hydro" in e2e_02).

3.2 Mode B (MilestoneCompaction): The Exact Save-Game JSON Checkpoint

In Mode B, as soon as Milestone 1 (collector.py) passed verification, the harness wiped the raw conversation history and started Turn 2 with this compact SAVE_GAME_COMPACTION_V1 JSON checkpoint:

{
  "checkpoint_type": "SAVE_GAME_COMPACTION_V1",
  "completed_modules": ["collector.py"],
  "history_compacted": true,
  "milestone_completed": 1,
  "next_target_module": "clearing.py",
  "verified_contracts": {
    "collector.collect_readings": "PASS (returns list of dicts with station_id, grid_zone in NORTH/SOUTH/WEST, mwh_units>0, rate_cents>=0, 0<=carbon_bps<=1500, non-mutating)"
  }
}

Because Turn 2 ran in a fresh 1,388-token context containing only the verified upstream contract and Milestone 2's specification, Mode B had zero attention dilution—properly implementing clean_tier = contract_tier.strip().upper() and passing all 20 / 20 test cases:

4. Practical Takeaway: Compact to Structured State, Not Prose Summaries

When building multi-step agent workflows (or using /compact in interactive coding agents):

1. Don't wait until the 200k context window is 95% full to compact: Attention degradation (Lost-in-the-Middle) starts happening well above 10,000–15,000 tokens of noisy tool logs. Compact at every milestone boundary as soon as a module's tests turn green.
2. Use deterministic JSON checkpoints instead of free-form prose summaries: As we saw in E05 (Subagent Folding), natural-language summaries often drop exact type contracts. Instead, record a structured JSON object containing completed_modules, verified_contracts (exact function signatures & invariants), and next_target_module.

🔗 Continue Reading the Series

• 🚀 Master Benchmark Hub (All 41 Patterns Ranked): I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy
• 🛡️ Methodology & Sandbox Architecture: How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox
• 🥇 Rank #1 Deep Dive (E20): Stop Your Coding Agent From Stepping on the Same Rake Twice: Cold-Start Exploration vs. a 5-Item MEMORY.md
• 🥈 Rank #2 Deep Dive (E21): Why Feeding Full Dependency Files to Your Coding Agent Hurts Accuracy: Full Source vs. Tree-sitter Interface Skeletons
• 🥉 Rank #3 Deep Dive (E16): Stop Dumping Raw Pytest Logs Into Your Coding Agent: How a Structured JSON Test Filter Cuts Input Tokens by 94.1%
• 🌿 Rank #4 Deep Dive (E19): Why Keeping Failed Code Attempts in Chat History Poisons Your Coding Agent: Error-Trace Pruning vs. Accumulated History
• 🔍 Rank #5 Deep Dive (E14): The Hidden "Schema Tax" of MCP Tools: Dumping 32 Tool Schemas vs. On-Demand Tool Search (-92.7% Input Tokens)