AI 編碼智能體在標準基準測試中接近頂尖分數,模擬職場任務完成率不足十分之一
AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace
一項把 AI 編碼智能體置於模擬軟件公司的實驗顯示,Claude Opus 5.5 和 GPT-6 Astra 完整完成的任務均不足十分之一。模擬環境包括同事、項目看板和聊天記錄;報道亦指出標準基準測試接近飽和,Anthropic 稱 Claude 編寫該公司超過 80 percent 的生產環境程式碼。

ChatGPT-5.6 prompted by THE DECODER
AI coding agents are nearing top scores on standard benchmarks. A new experiment puts them to work for the first time as digital employees inside a simulated software company with colleagues, project boards, and chat histories. Even Claude Opus 5.5 and GPT-6 Astra fully complete fewer than one in ten tasks.
Success stories about AI coding agents keep piling up. Anthropic says Claude writes more than 80 percent of the company's production code, and standard benchmarks like SWE-bench are nearly saturated.
來源:The Decoder · the-decoder.com