如何判斷 Agent 流量是否適合軌跡快取？Pomerene 列出衡量方法

How much of your agent traffic actually repeats? Measure before you cache

Trajectory caching only pays when your agent keeps solving the same problems. So before adopting it — ours or anyone's — measure whether your traffic actually repeats. This post is the checklist we'd want an engineer to run first, including the cases where the honest answer is "don't bother."

Start with the uncomfortable case

Pomerene saves little when every task is unique. If your agents do one-off bug fixing, where each ticket is a novel investigation, or bespoke per-customer work that never recurs in recognizable form, there is nothing stable to replay. A cache over unique traffic is a lookup table that always misses: you've added infrastructure, storage, and a matching step to every request in exchange for a hit rate near zero. We'd rather say that plainly than sell you a kit that can't help.

The workloads where trajectory caching earns its keep are the opposite shape: support agents fielding the same intents, booking flows, FAQ agents, scheduled jobs — anywhere the goal recurs even when the wording doesn't.

What to measure

You don't need new tooling to get a first read. Your existing traces and logs are enough for four numbers:

1. Goal repeat rate. Group completed runs by the goal they were trying to achieve — not the literal prompt, the intent. What fraction of this week's runs had a goal that also appeared last week? This is your ceiling. No cache can hit more often than your goals repeat, and if the repeat rate is under roughly a fifth of traffic, the economics get thin fast.

2. Paraphrase clustering. Raw string deduplication will lie to you. "Where's my refund?" and "I need an update on the money you owe me" are the same goal with zero tokens in common. Cluster requests by embedding similarity and look at cluster sizes: a long tail of singletons means unique work; a few fat clusters mean a small set of goals dominates your traffic, which is exactly the shape a trajectory cache exploits.

3. Cost per run. Sum tokens across all steps of a run, not just the final call — agent costs hide in the intermediate tool-use and reasoning steps. Multiply by your repeat rate and you have an upper bound on savings, in dollars, before you build anything.

4. Latency distribution. Record end-to-end run latency, p50 and p95. Long multi-step runs are where replay helps most: a fully replayed trajectory skips the entire step sequence, so the longer and more repetitive your runs, the bigger the latency win per hit.

What Pomerene reports once it's running

If you do deploy it, Pomerene ships diagnostics views so you can verify the cache is doing what the back-of-envelope math promised:

• Hit rates per tenant and global — whether the repeat rate you measured in traces shows up as actual hits, and whether one tenant's traffic behaves differently from another's.
• L1 / L2 / miss breakdown — full replays, partial replays (where your agent resumed at the step the cached plan stopped matching), and clean misses. A healthy deployment isn't all L1; a meaningful L2 share means the cache is absorbing the common prefix of your work even when goals partially diverge.
• Latency percentiles on the hit path versus live runs.
• Dollars and tokens saved, with accounting that matches how replay actually works: an L1 hit counts all of the trajectory's tokens (the whole run was skipped), while an L2 hit counts only the replayed fraction. No double-counting the steps your agent still executed.

Quality is monitored, not assumed. A /v1/feedback loop lets outcomes feed back into the cache: entries whose replay quality drops get evicted rather than served forever on the strength of one good run.

The honest limits

Two constraints worth knowing before you design around it. Pomerene currently runs as a single-writer instance with an in-memory vector index — simple and fast, but it's one process owning the cache, not a distributed store. And because stored trajectories are indexed by embedding, changing the embedding model invalidates stored vectors: swap models and the index has to be rebuilt rather than migrated in place. Neither is hidden in fine print; both should factor into whether this fits your deployment today.

The decision rule

Measure goal repeat rate and paraphrase clustering first. If a third or more of your traffic is goals you've already solved, trajectory caching will very likely pay for itself in tokens and latency — and you'll be able to prove it from the diagnostics within a week. If your traffic is genuinely one-of-a-kind work, skip the cache and spend the effort elsewhere. Either way, you'll have made the call from your own numbers instead of a vendor's benchmark.

Pomerene's self-hosted Starter Kit — Indie $99 / Pro $249, one-time, source included — comes with the diagnostics views described above, so the measurement doesn't stop at adoption. See pomerene.io/starter-kit.