跳到正文
The Decoder· Jonathan Kemper·· 39 分鐘前AI 評分52

AI 編碼智能體在標準基準測試中接近頂尖分數,模擬職場任務完成率不足十分之一

AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace

AI 導讀

一項把 AI 編碼智能體置於模擬軟件公司的實驗顯示,Claude Opus 5.5 和 GPT-6 Astra 完整完成的任務均不足十分之一。模擬環境包括同事、項目看板和聊天記錄;報道亦指出標準基準測試接近飽和,Anthropic 稱 Claude 編寫該公司超過 80 percent 的生產環境程式碼。

正文

Image description

ChatGPT-5.6 prompted by THE DECODER

AI coding agents are nearing top scores on standard benchmarks. A new experiment puts them to work for the first time as digital employees inside a simulated software company with colleagues, project boards, and chat histories. Even Claude Opus 5.5 and GPT-6 Astra fully complete fewer than one in ten tasks.

Success stories about AI coding agents keep piling up. Anthropic says Claude writes more than 80 percent of the company's production code, and standard benchmarks like SWE-bench are nearly saturated.

來源:The Decoder · the-decoder.com