跳到正文
DEV Community· Ilja (ilya) Nevolin·· 9 天前AI 評分53

智能體記憶的難點在於察覺內容何時已過時

The hard part of agent memory is noticing when it's stale

AI 導讀

Big Agentic Brain 評測顯示,當記憶中只有過時條目時,Sonnet 在 88% 的測試中察覺條目已過時,但只在 4% 的測試中將其標記,成功率回落至無記憶基準。

正文

Ilja (ilya) Nevolin

A lot of teams are now giving coding agents a shared memory that follows them from one tool to another. That's the right direction.

The hard part, at least in what we measured, is not storing the memory. It's noticing when it is no longer true. In our Big Agentic Brain evals, when the only entries in memory were outdated, Sonnet saw the stale one in 88% of those runs and flagged it in only 4%. Success fell back to the no-memory baseline.

So the company brain we built treats upkeep as part of the design: a correction supersedes the old entry, and an optional valid-until date archives it the day after. Memories do not expire by default; the date is for time-bound facts.

I wrote up the broader case here: https://nevolin.be/blog/company-brain-shared-board-for-agents

The code is here: https://github.com/enhesa/big-agentic-brain

I'm involved in building BAB.

來源:DEV Community · dev.to