我在系統提示詞加入時間戳，提示詞快取便從未命中

I did not know prompt caching existed.

I knew the API billed me per token, and I knew a long conversation cost more than a short one. What I did not know was that providers keep the processed prefix of a prompt around and charge a fraction of the price to reuse it. So when I looked at the usage line and it said zero cached tokens, I did not read that as a problem. I read it as a number I did not understand yet, in a bill I had already decided was the price of the model.

Then I updated Hermes, moved off my OpenRouter key to OpenCode, and cached tokens started showing up in the same place.

That is the whole reason I went and read the docs. Not because I wanted to optimize anything, but because a number that had always been zero suddenly was not.

The part I had done to myself

For every agent I build, I put the current time in the system prompt. It is one line:

Current time: 2026-11-04T01:17:00Z

I still think that is a reasonable thing to want. A model without a clock will confidently answer questions about "today" using whatever it saw during training. If the agent is supposed to help with a deadline or a schedule, it needs to know when now is.

The problem is that the line changes every request.

Prompt caching works on prefixes. The provider hashes the start of your prompt, and if it has seen that exact hash before, it can skip reprocessing everything under it. Change one character near the top and the hash changes, which means the entry does not exist, which means the whole prompt gets processed and billed again. My helpful clock line was sitting at the top of the prompt, in the part that everything else depends on.

The three things a hit needs

Once I understood the shape, the rest of the docs were easier.

A long enough prefix. OpenAI will not cache anything under 1,024 visible input tokens, and hidden system content does not count toward that minimum. Anthropic's newer models start at 512. Below the line, no setting helps you.

An identical prefix. Byte for byte, up to the point you marked. This is the condition my timestamp violated.

A marker in the right place. Anthropic documents the cache as a hierarchy: tools, then system, then messages. A change at one level invalidates that level and everything after it, so a volatile line at the top takes the whole prompt with it.

And a detail that reframed the whole thing for me: on OpenAI's supported models, caching is on by default. There was never a switch I forgot to flip. The only question was whether I was sending something the cache could match.

Never hitting is worse than not having it

Here is the part I had genuinely never read. A cache write costs more than a normal input token. It is not a discount you turn on. It is a trade: pay a little more to write, pay a lot less to read.

Which means a prompt that changes on every request pays the write price every single time and never collects the read price. I had assumed "caching enabled" could only help. It can hurt.

A model of a forty-turn loop, with a 6,000-token static block, 1,000 tokens appended per turn, 300 output tokens per turn, priced at $2 per million input and $10 per million output:

Same conversation, same answers. The worst case is a quarter more expensive than having no caching at all, and six times the cost of getting it right.

Those rates are my assumptions, not a quote from your invoice. The direction is the point.

What actually changed for me

Two things changed at once: I updated Hermes and I moved off the router. That is the worst way to debug your own setup, and I am not going to pretend I know which one did it.

What I can tell you is what I look at now, and what I would check first if the number went back to zero:

• Is the top of my prompt identical between requests? Tools, system prompt, and anything retrieved and inserted near the front.
• Is the prefix over the minimum for the model I am using?
• Am I going through a router that could put two requests on different providers?
• How long was the pause since the last request? Caches are measured in minutes.

The clock itself does not have to go. Put it in the last user message, or round it to the date if the agent only needs the day. The agent still knows what time it is, and the prefix above it stays the same.

The number I should have been watching

I spent a long time assuming my bill was the price of the model. It was the price of a prefix I kept changing, on a route I never chose.

If you cannot say what fraction of your input tokens came back from cache, you do not have a cost model yet. You have an invoice.

The longer version, with the source links for every default and the arithmetic behind the table, is on my blog: Caching That Never Hits Costs More Than No Caching.