CodeCook 如何減少長時間 Agent 工作階段中重複傳送的 tokens

If you've run an AI coding agent through a long session, you've probably seen how the cost grows. Turn 3 is cheap. Turn 30 isn't, even when all you ask for is a one-line fix. I've written about CodeCook before. This post is about the mechanics: what actually gets sent on every turn, what I decided is safe to stop sending, and where that trade-off still goes wrong.

What a "turn" really sends

A chat model has no memory between requests. Every time the agent takes a turn, the client sends the whole working context again:

• the system prompt and tool definitions
• every earlier user message and model reply
• every earlier tool result: file reads, directory listings, terminal output, test logs

That last group is the problem. A single file read or a noisy test run can be thousands of tokens. It helped the model once, on the turn it happened, and after that it's mostly dead weight. It still gets resent, and billed as input, on every turn that follows. So the cost of a session doesn't grow with the work you're doing now. It grows with everything that came before.

The approach: summarize the old, keep the recent

CodeCook is a VS Code extension, a coding agent in the sidebar, and its main job is to stop that resending. The idea is simple:

1. Older turns get compressed into a summary. Later turns send that summary instead of the full earlier text.
2. Bulky tool output gets compressed too, so a 2,000-line log from twenty minutes ago doesn't ride along forever.
3. Recent turns are still sent in full, because that's where the details the model needs right now live.

You can also trigger summarization yourself ("summarize a long conversation on demand") when you know you've finished one phase of the work.

Summarization uses its own model setting, codecook.summaryModel, separate from codecook.model for agent turns. That way you can use a cheaper model to write summaries and save the stronger one for the actual coding. Token budgets and tool-result limits are configurable too, under CodeCook in Settings.

The plan lives outside the summary

The biggest risk with summarizing is that the agent forgets what it was trying to do. A summary of "we discussed auth" doesn't help much at step 6 of a 10-step refactor.

So the plan is kept separately and survives summarization. You build it, you can reopen it from the command palette, and the agent keeps following it in long sessions even after the turns where it was written are compressed. Constraints like "keep the existing database tables" or "auth stays on the existing provider" belong in the plan, not in chat history that might get summarized away.

The code itself is another safety valve. The real state lives in your workspace files, not in the chat. If turn 7 needs a function written in turn 3, the agent can read the file again instead of relying on the conversation remembering it.

What the numbers look like (honestly)

In one long agent session, a single turn went from 800,787 tokens to 14,857 after compression, about 98% fewer, with the plan and the work intact.

To be clear about what that is: one turn, deep into a long session, and a best case, not an average. Savings depend on how long the conversation is, which model you use, how much tool output piled up, and when you summarize. A short session won't see anything like that.

To make this visible rather than a claim, the token panel compares "history if sent in full" with "history actually sent" for each turn, and the savings report adds that up. Those token and cost numbers are estimates, not your provider's bill. Client-side token counts can drift from what the server counts, so the usage your provider reports is the real number. Checking those estimates against actual billed cost across many sessions is the next thing I'm measuring.

Spend caps, because estimates still help

Since the per-turn cost is estimated anyway, CodeCook can stop a session from running away. You can set daily, weekly, or monthly caps:

{
  "codecook.spendLimitDailyUsd": 5,
  "codecook.spendLimitWeeklyUsd": 20,
  "codecook.spendLimitMonthlyUsd": 60
}

0 means no cap. Per-million-token prices are configurable, so the estimates can match the model you actually use.

Checkpoints: the other half of trusting a summary

A summary can leave details out. I'd rather admit that and make mistakes cheap to undo. Every turn is checkpointed. You see every file the turn changed, can open one diff or all of them in VS Code's native diff view, keep all, revert all, or rewind to an earlier turn and restore your files. If a compressed context leads the agent somewhere wrong, you roll back that turn instead of untangling it by hand.

Setup, including the sign-in step

• Bring your own key. Google Gemini (Vertex AI with gcloud login, or a Gemini API key), OpenAI (API key or your ChatGPT plan), Anthropic Claude, Azure OpenAI, or AWS Bedrock. You only need one. You pay the provider directly, with no markup. Vertex AI is the default; set codecook.gcpProject if you use it.
• There's a sign-in step. When you first open CodeCook, you sign in with Google or GitHub. That creates a CodeCook account stored only in your local VS Code install. It's separate from your model provider and gives CodeCook no access to your repositories. GitHub sign-in uses the device-code flow. I know any sign-in is friction, so tell me if it stopped you.
• Requires VS Code 1.85+. MIT-licensed and free.

Where it still falls short

• It isn't symbol-level dependency tracking. If a detail lived only in an old turn and never made it into a file or the plan, the summary can lose it.
• Savings vary a lot; the 800K → 15K turn is not typical.
• Cost numbers are estimates until I've finished checking them against real bills.

Try it

• VS Code Marketplace: https://marketplace.visualstudio.com/items?itemName=makayis.codecook
• Open VSX (Cursor, VSCodium and other VS Code-compatible editors): https://open-vsx.org/extension/makayis/codecook
• Website: https://www.codecook.makayis.com

If you try it on a long session, I'd really like to hear two things: where a summary dropped something you needed, and how far the estimate was from what your provider billed. Those two answers decide what I work on next.