Parsimony：开源指标，评估 AI 编码模型添加的代码量增减

Parsimony 是一个开源的评估工具，用于衡量现代 AI 编码模型（如 GPT 系和红-team Claude）在处理代码任务时添加的多余代码量。它基于现有代码生成 (code generation) 基准数据，包括 DeepSWE-GPT-3.5系, SWE-bench Verified, 和 SWE-bench Live^\+。细节：

• The problem: Most benchmarks measure solve rate and token output or cost but do not punish for additional code, making a codebase harder to work with in the long term. Parsimony measures units (statements, calls, names and literal values) to count net changes and added code. I did not use lines of code for the usual reason, cramming things together doesn&#x27;t automatically make the codebase smaller or better.<br>

​ \<span class="mark">\\</span> Net units added across tasks on DeepSWE-GPT-3.5, SWE-bench Verified, and the smaller SWE-bench Live<br> GPT-6 and Claude 5.5 have no public export yet, I ran them myself on 3–10 tasks on my own subscriptions (faded circle). How Parsimony works Code count method: Transpile to the same major language (Python in this case) Token-based with Treesitter grammar for accurate syntax parsing, so it&#x27;s language-agnostic Aggregate counts per file type Count added (“insertion”), removed (“deletion”), and net changes in: Statements like def or for Calls like logger.info or pandas.read_csv Names like function or variable names Literal values like strings or numbers líder completa em português no README Any feedback or model contributions are welcome, especially public patch exports for First-Party models (<a href="https://github.com/La5u/Parsimony/issues" rel="nofollow">Contribute</a> | <a href="https://github.com/La5u/Parsimony#chisuguro-models" rel="nofollow">Add a Model</a>) Not affiliated with any of the benchmarks.