作者一夜下調個人 AI 的 2,439 條記憶信心評級，系統變得更聰明

When we audited the memory store behind my personal AI, we found 2,466 active memories — and every single one carried a confidence score of 1.0, the top trust grade. Not most of them. All of them, including summaries the system had written about its own summaries. None of that confidence had been earned; it had been assigned by an import pipeline as a default. So we downgraded 2,439 memories in one night. The system did not lose knowledge. It lost a lie it had been telling itself about how sure it was.

Here is the uncomfortable question nobody asks about AI memory: not "what does it remember?" and not "how much can it recall?" — but "where did each memory's confidence come from?" In most systems I have seen, the honest answer is: nowhere. The confidence field exists. It is filled in. It is just filled in by whoever — or whatever — wrote the row, at whatever value the writing code happened to default to. Then every downstream reader treats that number as a measurement.

A perfect score nobody earned

Our audit counted 2,626 rows in total: 2,466 active memories and 160 superseded ones we left untouched. The active set was unanimous in a way real knowledge never is. Every confidence score was 1.0. 98.9% of active memories — 2,439 of them — sat at the highest trust grade, the grade that is supposed to mean "verified against a primary source."

They had not been. Most had arrived in two bulk imports: notes, files, and compiled summaries loaded in batches. Worse, 27 of them were derived summaries — text the system itself had condensed from other memories — and they carried the same perfect grade as firsthand records, displayed to me with the same "verified" marker. A summary of a summary, wearing a medal it never won. That is not a memory system with high confidence. That is trust laundering: each pass through a pipeline washes away provenance and leaves the claim looking cleaner than it entered.

The mechanism mattered more than the numbers. Our normal write path grades a memory by its source — what the person stated, what a document proves, what the system merely derived. The import pipeline did not use that path at all. It wrote directly, at the top grade, bypassing the review every ordinary memory goes through. The most trusted records in the system were precisely the ones that had never been checked.

Storage is not verification

The popular view of AI memory is a capacity problem: longer context, bigger stores, better retrieval. Bolt on a memory layer, import your notes, and your agent "knows you." I now think that framing is exactly backwards. Importing a claim proves one thing only — that the claim was copied successfully. It says nothing about whether the claim is true, current, firsthand, or even attributable. A system that cannot distinguish "he told me this himself" from "a batch job found this in a compiled file" does not have a memory problem. It has an epistemology problem with a database attached.

And the damage is quiet. A low-trust memory that knows it is low-trust gets checked before it drives a decision. A low-trust memory mislabeled as certain drives the decision directly — and when it is wrong, nobody can reconstruct why it was believed, because the record says it was verified. False certainty is worse than ignorance in a system that acts. Ignorance asks. False certainty proceeds.

The downgrade is the feature

So we regraded everything, in four batches that reconciled to the row: 31, then 332, then 896, then 1,207. The rules were deliberately unglamorous. Imported material defaults to unconfirmed — it can be read, searched, and used as a lead, but it cannot present itself as fact. Derived summaries are capped at the lowest grade: a summary can point back to evidence, it can never outrank it. A person's own direct statements sit one grade higher, because testimony is evidence, just not proof. The top grade is reserved for claims tied to a primary credential — and nothing gets it by default, ever again. Imports now go through the same grading entry as every other write; direct writes at the top grade are banned by construction, not by convention.

The final distribution looks, at first glance, like a collapse: 2,437 memories at the lowest working grade, 27 derived at the bottom, 2 in the middle, and zero — zero — at the top. I consider it the healthiest the store has ever been. Those numbers are not a verdict that the memories are false. Most of them are probably true. They are a verdict that truth had not been demonstrated yet, record by record, and that the system will now say so. A memory can earn its way up when a primary source shows up. Until then, it waits at the grade it deserves.

If you run any AI system with a memory store, run this audit before you add another feature: count your rows by confidence, and ask what fraction of the top grade can produce its evidence on demand. If the answer is "the import gave it that score," you do not have a knowledgeable agent. You have an agent that is certain about things it merely copied — and certainty, unearned, is the most dangerous capability you can ship.

— Jiahui Miao, founder of vertciti, building a personal AI operating system for one.