跳到正文
DEV Community· Luke Wilcox·· 3 小時前AI 評分72

AI 編碼工具增加程式碼,不等於交付更多產品

More AI code isn’t more product

AI 導讀

Roiva 指出,AI 編碼工具帶來的程式碼增量,並未等比例轉化為完成工作或已發佈的軟件。企業採用編碼 Agent 後,開啟的 pull request 增加35.7%,完成的 Jira 工作項目增加13.7%;開源開發者的程式碼行數變更量增加958%,發佈量則增加20%。企業程式碼由提交審核至獲批的平均時間增加56.1%;文章建議先檢視審核能力,再考慮增加席位。

正文

Roiva wrote this, and Roiva's formulas read coding-assistant and Jira data. It first appeared on Roiva's field notes on November 2, 2026, and each source was read that day.

As developers have known for a long time, more code is not better. More code means more review, more maintenance and more chances for error. AI coding tools are testing this lesson at scale.

Two recent studies follow the gain from AI coding tools stage by stage, from lines written to software shipped, and most of it doesn't make it to the end. That matters for a decision budget owners face now: when a team has good activity numbers and wants more seats or more tokens, do you say yes?

The same gap shows up across whole companies. In McKinsey's latest survey, 80% of respondents said AI improved their productivity, while 37% said AI had added anything to their organization's EBIT, about the same as a year earlier. A survey can show that gap, but it can't show where the gain goes. Software can, because each step from code to release leaves a record.

How much of an AI coding gain reaches finished software?

Much less than the code numbers suggest. At companies, adding coding agents raised the number of pull requests opened by about 36% and the number of work items finished by about 14%. Among open-source developers the gap was wider.

The company figures come from Chen and Stratton, a Harvard working paper using Jellyfish data on 720,542 workers at 655 firms. The open-source figures come from Demirer, Musolff and Yang, at MIT, Wharton and Boston University, who studied more than 500,000 GitHub developers. A sync agent is a tool such as Claude Code that works alongside a developer, who reviews and commits what it writes.

Stage Companies adopting agents Open-source developers adopting sync agents
Code written Lines added +44.0% Lines changed +958%
Commits +36.0% +153%
Pull requests Opened +35.7% Created +86%
Finished work Jira issues resolved +13.7% Releases +20%

Read each column from top to bottom, not across. The company figures are firm-wide and come on top of what earlier coding assistants added. The open-source figures are for the individual developers who adopted a tool, which is one reason they are so much larger. The 13.7% is borderline statistically significant, and Chen and Stratton rule out a gain above 30.7%.

Demirer, Musolff and Yang also cite Meta's reported numbers, which follow the same pattern: internal code changes up 220% in a year, and changes reaching users up 36%.

Where does the extra code go?

At companies, much of it waits for review. After firms adopted agents, the average time from submitting code for review to approval rose 56.1%.

Code waited in the backlog 59.9% longer and spent 58.5% longer in active review, and each code change drew 38.6% more comments. Chen and Stratton read the longer wait as limited review capacity, and the longer, more heavily commented reviews as a sign that submitted code got worse.

The paper doesn't say how much of the increase in comments comes from AI reviewers. AI wrote 23.3% of all review comments while appearing on only 10.8% of pull requests, so where AI reviewers show up, they comment a lot. We see the same on our own pull requests.

People still do most of the reviewing, and that may be a deliberate choice: Chen and Stratton suggest firms may keep people in review "given the high cost of deploying faulty code."

Is the extra code worth shipping?

Neither study can say, because neither measures what the finished work was worth. What they do show is that much of the extra code never ships.

Among open-source developers, the authors attribute most of the drop-off to experimentation. After adopting sync agents, developers added 7.0 times as many lines, deleted 12.2 times as many, and created 79% more branches. The authors say abandoned work isn't necessarily bad work, because people are choosing among attempts. Those attempts still cost tokens and developer time, so they belong on the cost side of an ROI calculation.

At companies, the work items that did get finished didn't get smaller: Chen and Stratton found no change in how long each issue was predicted to take. So the 14% isn't a count inflated by thinner tickets. Whether those items were product improvements or low-value edge cases is a question for the people who own the product.

Shipping isn't the last stage either, because new software also has to be used. Demirer, Musolff and Yang checked this on software marketplaces. New iOS apps ran at 33,000 to 45,000 a month through early 2025, then reached about 108,000 by April 2026. Ratings on each month's new apps stayed roughly flat from 2024 to 2025, and the share of new iOS apps with fewer than 10 ratings in their first three months rose from about 78% to 87%. This is consumer software, ratings are a rough measure, and the authors can't tell low quality from slow discovery. We take it only as a reason not to stop counting at "shipped."

What does the next AI dollar buy?

That depends on which stage is holding up the work. If the limit is review, more seats mostly add to the queue.

Neither study measures dollars or tests buying more seats, so this is our reading of them. Chen and Stratton's model gives a sense of the size. With coding tools continuing to improve, the model has finished software up 57.8% by 2031 if review productivity stays where it is, and output levels off there. If review productivity keeps pace with coding, finished software rises 341.4%. These are model scenarios and not forecasts, and engineering employment falls in both, by 10.3% and 76.2%.

Most teams we talk to aren't ready to give AI agents more review and sign-off authority. Until they are, when a team asks for more seats because code output is up, we would look at review first. Spending on review capacity, review efficiency and quality, or better tests could return more than another round of licenses.

This doesn't mean spend less. McKinsey's high performers are the 6% of respondents who attribute at least 5% of EBIT to AI and call the value "significant." They are more than twice as likely as others to put over 15% of their technology budget into AI, and twice as likely to be scaling coding agents. Nearly three-quarters of them report fundamentally redesigning workflows, against one-quarter of everyone else, and they are twice as likely to have defined processes for measuring impact. That group is 92 respondents describing their own companies, so it can't tell you which of these things causes which.

How do you find where an AI rollout is held up?

Line up the numbers for each stage, from the first thing the AI touches to the last thing finance could see, and look for where the gain falls off. Do this before you approve more spend.

  1. Write out the stages for the work. The table below has some examples.
  2. Get a before and after for each stage, with the same scope and period on both sides.
  3. Find the biggest drop and check whether work is piling up there. A queue points to a capacity problem. No queue and a lot of abandoned work points to experimentation, or to output nobody wanted.
  4. Make the call and write down why, with the numbers as they stand that day.
Work Early stage Middle stage Last stage
Software Lines of code, commits Pull requests merged Issues resolved, releases, product usage
Customer support Replies drafted Tickets resolved without a person Lower staffing or vendor cost
Sales Emails sent Meetings held Deals closed
Contract review Drafts produced Contracts approved Days to signature

Only the software row comes from these studies. Chen and Stratton argue that the same structure applies wherever one person implements and another reviews, but their data is software, and Demirer, Musolff and Yang warn that their estimates may not carry over to other kinds of work.

What you see What to do next
The gain reaches the last stage Scale it
The gain drops at one stage and work piles up there Fix that stage before buying more
The gain drops and nothing piles up Hold spending until you know whether the extra output was wanted
The last stage doesn't move after a fair trial Cut it, or change what it's for

What don't these studies show?

They don't give you an ROI figure, because neither one measures revenue, cost or earnings.

Both are about software. Chen and Stratton's paper is a job market paper using data from Jellyfish clients who agreed to take part. Demirer, Musolff and Yang cover open-source work, with little coverage of enterprise software. Both are early, and the tools are improving quickly. The authors of the open-source study say the gap may narrow as tools reach later stages. The randomized trials of AI coding tools, METR's and the Copilot field experiments at Microsoft and Accenture, are set out in What the AI ROI research actually measured.

McKinsey's figures come from The state of AI in 2026, a survey of 1,719 respondents, and they are self-reported. Of those who answered the productivity question, 72% were C-level or senior executives.

Each source was read on November 2, 2026.

Where Roiva fits

Roiva keeps an initiative's costs beside its approved value, measured against a baseline, and reports cash and freed capacity separately. Roiva also records each decision to continue, scale, fix, pause or retire with the numbers as they stood that day.

Roiva's formulas for coding assistants read early and middle stages, such as accepted suggestions or merged pull requests, and report the result as freed capacity, not cash.

Roiva's formulas also read later stages, such as issues resolved in Jira and production deployments. Most of those report freed capacity as well; the failed-deployment formula reports avoided cost.

The method is published at roiva.ai/methodology.

來源:DEV Community · dev.to