Pomerene 解釋語義快取重播錯誤指令的風險與寫入防護

Your semantic cache will happily replay the wrong command

Semantic caching works well for LLM responses because a near-match is usually good enough. Ask a slightly different question, get the cached answer, save a call. The failure mode is a mediocre answer.

Trajectory caching — caching the sequence of tool calls an agent makes, not just its final text — has a different failure mode. The cache doesn't store prose. It stores actions. And a near-match on actions can be the opposite operation.

Concrete example. An agent runs the goal "deactivate user 4471" and the trajectory gets cached. Later, a request comes in: "activate user 4471". With text-embedding-3-small, those two goals score 0.853 cosine similarity. A plain semantic cache running a 0.85 threshold — a common default — scores that as a hit and replays the deactivation trajectory in response to an activation request.

The embedding didn't fail. The two sentences are on the same topic, mention the same entity, and have the same structure. Embeddings are good at topic. The token that flips the operation — the polarity — barely moves the vector. If your cache decides hits on vector distance alone, it will eventually serve a write it shouldn't.

The asymmetry that matters

For agent trajectories there are two directions a fuzzy match can go wrong, and they are not equivalent:

• A cached write trajectory fuzzy-matches a different request. This is the dangerous direction. The system serves actions the requester never asked for.
• A cached read trajectory fuzzy-matches a write request. This returns a wrong answer — but not a destructive one, because in Pomerene cached steps are never re-executed. A hit returns the recorded results of the stored steps. Nothing runs. No tool is called. The user gets stale read data presented as an answer, which is a quality bug, not an unwanted write.

We call the first case an operational false positive: a fuzzy hit that would serve a trajectory whose effect differs from what was requested. That's the metric a trajectory cache should be designed around — not raw hit rate.

The gate: writes are exact-only

Pomerene is a deterministic trajectory cache, and its matching rule is built around that asymmetry:

1. Every stored tool step declares its side effect. Each step in a trajectory is tagged read, replay_safe_write, or external_write. If a step doesn't declare one, the store API rejects the trajectory. You can't opt out of the classification by forgetting to annotate; unclassified trajectories don't get stored.

2. Any trajectory containing a write step is served only on an exact goal+context repeat. Fuzzy (embedding-based) matching still applies to read-only trajectories, where a near-match costs you a suboptimal answer. The moment a trajectory writes, it leaves the fuzzy path entirely. A paraphrase of a write goal is a miss, full stop — the agent runs live.

Illustrative store call:

curl -X POST localhost:8080/v1/store \
  -H 'content-type: application/json' \
  -d '{
    "goal": "deactivate user 4471",
    "steps": [
      { "tool": "users.get",        "side_effect": "read",
        "args": { "id": 4471 } },
      { "tool": "users.deactivate", "side_effect": "external_write",
        "args": { "id": 4471 } }
    ]
  }'
# This trajectory now serves ONLY on an exact repeat of goal + context.
# "activate user 4471" — 0.853 similar — is a miss, and runs live.

What the eval shows

We ran an adversarial eval of 18 near-opposite command pairs — activate/deactivate-style goals designed to sit close together in embedding space — using text-embedding-3-small at a 0.85 threshold:

One in eighteen sounds small until you price it: that's one wrong write served, silently, out of every eighteen similar command pairs that pass through the cache. Writes are exactly the operations you can't undo with an apology.

The honest tradeoff

The gate is not free, and we won't pretend otherwise. On these same deliberately short commands, the gated cache's paraphrase recall was 1/18 — low. That's partly the test being adversarial (two-to-four-word commands carry almost no signal beyond the verb, and the verb is exactly what the gate refuses to guess at), but the direction of the tradeoff is real and deliberate:

• A miss costs you one live agent run — the thing you were trying to cache in the first place.
• A wrong write costs you an incorrect mutation that may not surface until much later.

Pomerene is tuned to pay the first cost and never the second. If your workload is read-heavy research and summarization, a looser cache may serve you better. If your agents mutate state, the recall you lose to the gate is the price of being able to trust a hit.

Where it sits

Pomerene wraps your agent function, not your model client, and never touches your prompts. On the hit path there is no LLM involved at all — matching is hashing plus vector geometry. Three outcomes per lookup:

• L1 — full replay: the whole trajectory serves from cache. A 14-step trajectory replays in 47ms vs. 3.9s live.
• L2 — partial replay: a prefix matches; when the trajectory drifts, control hands back to the agent to finish live.
• Miss: the agent runs normally, and the new trajectory can be stored.

Because it wraps the agent rather than a framework's internals, it works with LangGraph, Vercel AI SDK, Mastra, or a custom loop. Quickstart is Node 20+ and Postgres; npm run setup gets you running in about five minutes.

Try it

Pomerene ships as a self-hosted Starter Kit — Indie $99 / Pro $249, one-time, source included. If your agents perform writes, run the near-opposite test against your current cache before you trust it in production.

pomerene.io/starter-kit