Continual Learning Exam 如何評測 LLM Agent 的長期 RSI 與經驗學習
Continual Learning Exam: Benchmarking Long-Horizon RSI and Experiential Learning in LLM Agents
Continual Learning Exam(CLE)推出一項基準測試,評估 LLM Agent 能否累積過往經驗、遷移所學知識,並在長期任務中持續學習。CLE 涵蓋 89 種職業的 120 項真實專業任務,共設 600 個逐步解鎖的關卡。17 種前沿 Agent 配置中,表現最佳的 Claude Code + Opus 5 完成 47.7% 的關卡及 23.3% 的完整任務梯級。
🚀 Introducing Continual Learning Exam (CLE): Benchmarking Long-Horizon RSI and Experiential Learning in LLM Agents!
Can LLM agents truly learn from experience and continuously improve over time?
While existing benchmarks primarily evaluate agents on isolated tasks, CLE shifts the focus from task solving to continual learning: Can agents learn from past successes and failures, accumulate experience, and transfer acquired knowledge, strategies, and skills to future tasks?
To evaluate this, CLE introduces game-like Task Ladders, where agents must complete each level to unlock the next, with later levels building on earlier experience.
Built with 50+ domain experts, CLE features 120 real-world professional tasks, 600 progressive levels, and 89 occupations, supported by ~24K files and 19 stateful applications.
📊 Across 17 frontier agent configurations, even the strongest, Claude Code + Opus 5, completes only 47.7% of levels and 23.3% of full Task Ladders.
These results suggest that even frontier LLM agents still have substantial room for improvement in long-term experience accumulation, knowledge transfer, and continual self-improvement.
CLE aims to provide a realistic and verifiable benchmark for advancing research on Agent Continual Learning, Memory, Skill Learning, and Recursive Self-Improvement (RSI).
💡 Beyond Task Solving: Towards Continual Learning Agents That Learn, Adapt, and Improve Through Experience.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co