跳到正文
Hacker News:AI 熱帖·· 2026-10-14AI 評分42

Show HN: Parsimony – an open-source benchmark for how much code AI models add

Show HN: Parsimony – an open-source benchmark for how much code AI models add

AI 導讀

Parsimony 是一項開源基準測試,透過計算補丁中的淨「coding units」,衡量 AI 模型增加的程式碼量。測試沿用 DeepSWE、SWE-bench Verified 和 SWE-bench Live 公開的補丁,統計語句、呼叫、名稱及字面值等語法元素,並扣除刪除的程式碼,不以行數計算。

正文

Existing benchmarks measure solve rate and token output/cost, but don't punish excess code, which makes a codebase harder to work with in the long term. I wanted to measure this since it matters most for long-term projects.

Method: it takes the patches that benchmarks already publish (DeepSWE, SWE-bench Verified, SWE-bench Live) and counts "coding units": syntax elements like statements, calls, names and literals. I didn't use lines of code, so cramming code isn't rewarded. Deleted code is subtracted, so it measures net units.

GPT-6 and Claude 5.5 have no public patches yet, so I ran them myself on 3–10 tasks on my own subscriptions (shown as faded circles, not ranked).

Any feedback or model contributions are welcome, especially public patch exports for newer models.

GitHub: https://github.com/La5u/Parsimony

Not affiliated with any of the benchmarks.

來源:Hacker News:AI 熱帖 · news.ycombinator.com