Atlarix 修正 Build 模式提示詞快取錯誤，避免每則訊息重寫約 19,000 個 token

Prompt caching is the cheapest win in an AI coding agent. The system prompt and tool definitions are the same on every request, so the provider can keep them and charge a fraction for reading them back. Atlarix has marked its prompt for caching for a long time, and the logs looked right. Then a QA pass compared two messages in the same chat, byte for byte, and found that in Build mode every message after the first rewrote the whole cache instead of reading it.

How the cache works, briefly

Claude's cache matches on an exact prefix: tools, then the system prompt, then the conversation, up to a marked breakpoint. If any byte before the breakpoint changes, everything from that byte on is a miss and is written to the cache again. On Anthropic's pricing, writing costs more than normal input (1.25× for a five-minute entry, 2× for a one-hour one) and reading costs about a tenth. So a cache that is written every time is worse than no cache at all.

What we were sending

Atlarix builds its system prompt in two parts. The stable header holds who the agent is, the tool rules and the mode's instructions; it is the same for the whole chat, so it carries the cache breakpoint, with a one-hour lifetime. The per-turn tail holds today's date and this message's request; it changes every message, so it is meant to go after the breakpoint, in a block of its own.

Ask mode did exactly that. Build mode joined the two parts into one string, then appended the context sections, the guides and project notes, after it. So the request sat in the middle of the one block that carried the breakpoint.

Why nothing looked wrong

Inside a single message, the agent makes several model calls as it works through tool results, and those all share the same system prompt. So the logs showed healthy cache reads: on step two of a message, 19,379 tokens read from the cache. The miss only showed between messages, where the request text changes. A check that looks at one message at a time will always pass.

How we found it

During QA for the Anthropic-format providers, we captured two consecutive request bodies from the same chat and compared them. The first difference was at character 22,966 of a 24,000-character system block: the <user_context> holding the user's request. Everything after it was identical, and everything before it was identical. That one block was the entire difference.

The fix

The finished Build prompt is now split back into its two parts before it is sent: the cached part, which no longer contains anything that changes per message, and the per-turn tail, sent as its own system block after it. No words in the prompt changed; only where the tail sits. Every other part of the app that reads the prompt still reads it whole.

The test that guards it does what the QA pass did: it runs two messages and requires the cached block to be byte-for-byte identical, with the request in a later block. On the old code it fails; on the new one it passes.

Before and after, measured

Three messages in one chat, Build mode, Claude Haiku 4.5 on Amazon Bedrock:

From the second message on, the fixed part of the prompt is now a cache read at about a tenth of the input price, where it was a one-hour cache write at twice the input price. For the fixed part of each message, that is roughly fifteen times cheaper on Claude, and the same layout is what lets any provider that caches by prefix reuse it.

The rule we take from it: anything that changes per message must come after the last cache breakpoint, and the only reliable check is to compare two real requests, not one. The fix shipped in Atlarix 15.9.

Originally published at atlarix.dev.