MemSteer:將用戶歷史記錄轉化為監督訊號,以最佳化智能體記憶
MemSteer: Turning User Histories into Supervision for Agent Memory Optimization
北京大學與 Microsoft Research Asia 開發 MemSteer,將用戶歷史記錄轉化為測試,以最佳化 AI 智能體記憶。在 ATM-Bench 上,MemSteer-Opt 將 BM25 基線的 Question Score 從 54.9 提升至 68.3,使用官方 benchmark QA 時則達 67.3。
🧠 You set a deadline by email, then moved it in chat. Which date will your AI assistant remember?
We built MemSteer at Peking University and Microsoft Research Asia to turn a user’s own history into tests for agent memory. It connects facts across records, writes a question that requires putting them together, and keeps it only if a second reader can independently recover the answer and cite the evidence.
When the agent gets a question wrong, MemSteer-Opt uses that evidence to trace the failure and test changes to how the agent stores, retrieves, and presents information. No hand-labeled questions or model fine-tuning required.
🔍 On ATM-Bench, MemSteer-Opt took a BM25 baseline from 54.9 to 68.3 in Question Score. The same optimizer reached 67.3 using official benchmark QA.
📬 On RHELM, accuracy improved over BM25 by 4.1 points with chats alone and 6.6 points when emails and documents were included.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co