你的 AI 還記得舊答案：如何處理已改變的事實？

I work at Wontopos, so I'm biased. I build a memory API. But this post isn't about it. It's about a problem I keep reading about, and I want to hear how you deal with it.

The problem

An AI that remembers things will sooner or later remember something that is no longer true.

Here is a made-up example, just to show the shape of it:

Both memories are true statements. One of them stopped being true. If the memory layer only looks for text that is similar to the question, both come back, and the model has to guess which one wins. Sometimes it guesses wrong, and it sounds confident when it does.

That's the part that bothers me. A missing memory is easy to notice. An old memory that sounds right is not.

Why it's harder than it looks

• Similar is not the same as current. Search finds what matches the question. It doesn't know which fact replaced which.
• Adding more memory makes it worse. Every correction is one more thing that can contradict an older entry.
• Nobody tells the system. People don't say "forget what I said in March". They just mention the new thing.
• The failure is quiet. Nothing crashes. The answer is just a little wrong.

What people do about it

From what I've read, people seem to do one of these:

1. Edit the notes file by hand.
2. Tell the AI "that's wrong, now it's X" and hope it overwrites.
3. Wipe everything now and then and start again.
4. Store everything with timestamps and let the model sort it out.

What we do

In our API, an outdated memory is replaced on purpose with a supersede call, so the old value is marked as replaced and not just buried. We also keep revisions, so you can see what changed. I don't want to oversell this. Deciding when to call it is still your job, and I'm still thinking about how to do that part better.

We also put a STALE axis in Glasshouse, an open memory benchmark I'm working on (still a draft), because I couldn't find an easy way to measure this. Details are in the repo if you want to look.

My question

I'd like to hear from people who build or run their own memory:

• Has your agent ever answered with something that used to be true? What was it, and how did you notice?
• How do you decide a memory is outdated? Timestamps, a rule, a human, the model itself?
• Do you overwrite the old value, or keep both and mark one as old?

You don't need to sign up for anything. The details are at wontopos, and the site has a short survey on the main page. A comment here is just as good, and the survey is a bonus.