跳到正文
DEV Community· Paragon·· 5 天前AI 評分38

SWE-CoWork 基準測試在模擬組織中評估編碼智能體協作能力

SWE-CoWork tests coding agents as collaborators in simulated organisations

AI 導讀

研究人員推出 SWE-CoWork,透過模擬組織中的協作任務,評估編碼智能體處理真實開發工作的能力。基準涵蓋 101 個真實程式碼庫、11 種語言及 12 個領域;智能體須向 26 名模擬同事和記錄查找需求,並協作及核實工作。模擬同事是否足夠真實,仍是評估結果能否反映真實組織表現的不確定因素。

正文

Paragon

Researchers introduce SWE-CoWork, a benchmark of 101 development assignments across 101 real repositories, 11 languages, and 12 domains. Agents work in a simulated organisation with 26 colleagues, recover requirements from people and records, and must collaborate and verify their work.

Why it matters

Real coding work is collaborative and underspecified; benchmarks that test only isolated code generation may overstate workplace readiness.

Analysis

SWE-CoWork addresses a gap between isolated coding tests and workplace practice by adding distributed requirements and colleagues. That should expose planning, communication, and verification failures. Main uncertainty: whether simulated colleagues behave realistically enough for results to predict performance in real organisations.

Source: arXiv

AI disclosure: this article was produced with AI assistance and reviewed for accuracy.

來源:DEV Community · dev.to