🔍 Read the full analysis: Where AI Coding Pays Off: A Framework For Measuring Value on ThorstenMeyerAI.com
Get audio and creator gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A report drawing on interviews at 27 workplaces and data from AI coding vendors finds that spending and productivity results vary widely across engineering teams. AI tools appear most useful for tasks such as issue triage, code archaeology and backlog work on uniform systems, but the evidence does not establish a single productivity gain. The report recommends tracking accepted changes, rework, failures and completed work rather than token use or raw pull-request counts.
Engineering teams are spending between $150 and about $3,000 per developer each month on AI coding tools, but available evidence does not show a consistent return across companies, according to a review published October 23 by ThorstenMeyerAI.com. Drawing on interviews at 27 workplaces and vendor and research data, the review argues teams should measure accepted work, rework and delivery outcomes, rather than treating tool usage or pull-request counts as proof of value.
The review builds on an October 20 report by Gergely Orosz of The Pragmatic Engineer, based on conversations with engineering leaders and senior engineers at 27 workplaces, alongside aggregate data from code-review vendors CodeRabbit and Gitar. The source article says its discussion of Orosz’s work uses information from the freely readable portion. Reported budgets varied widely and did not appear to track company size consistently: one public company with 1,500 engineers set a $150 monthly cap, while the two-person European agency Ix Technologies reported spending about $3,000 per developer.
Productivity figures also diverge. CodeRabbit reported that developers on its platform created 113% more pull requests per week than a year earlier, while a Microsoft study of tens of thousands of engineers found adopters merged about 24% more pull requests over four months than they otherwise would have. These figures have different populations and methods; they do not establish a general effect for all engineering teams. Microsoft’s study authors cautioned that a merged pull request is not necessarily valuable work. Several companies interviewed reported flat pull-request counts, even as some saw larger changes or more work completed.
The review identifies issue triage, code archaeology, prototypes and backlog cleanup on simple, consistent technology stacks as areas where teams reported practical gains. A staff engineer at a SaaS company with about 600 engineers said customer-issue triage that could take a day had become minutes of agent work plus up to an hour of human checking. Ix Technologies, which builds in C# and Blazor, reported clearing older backlog work. By contrast, teams working with complex legacy code may see less benefit, and the source warns that larger pull requests can burden review and security checks.
Infographic 01 · Where the money goes
From a $150 cap to $3,000 a month per developer.
Anecdotal data from 27 workplaces, mostly anonymized. It shows how wide the range is, not an industry average.
Spend per developer per month
Examples from the free part of the article · scale to $4,000
actual or typical spendbudget cap
What changed in 2026
Why the 2027 budget needs a range
- Peak in June and July: spend has roughly halved since for companies on API plans (Opus 5.5 and open models).
- Caps are common at medium and high spenders, and most people say they are easy to get around.
- Subsidies are ending: some companies must optimize or see the bill rise 5–10×.
- GitHub Copilot bills by token since 1 June 2026, at each model’s API rate, with no fallback to a cheaper model.
- Company size does not predict spend.
ROI is “a question that few companies seem able to answer – at least for now.”Gergely Orosz, The Pragmatic Engineer, 20 Oct 2026
Sources: The Pragmatic Engineer, “What’s the ROI of software engineers’ spending on AI?” (20 Oct 2026, free part); GitHub (27 Apr 2026).
Powered by Thorsten Meyer AI
Budgeting Beyond Tool Usage
The variation matters as companies set 2027 engineering budgets and some discounted subscriptions approach expiration. The review says several companies expect those subsidies to end, potentially exposing them to bills that could rise five- to tenfold if they do not change plans or usage. It also reports that API-plan spending at some firms peaked in June and July and had roughly halved since, a change Orosz attributed to the release of Opus 5.5 and a shift toward open models. Those are reported patterns, not a forecast for every customer.
Tool adoption can create costs beyond subscriptions. If a team produces more or larger pull requests but does not have enough review capacity, the extra output may not translate into reliable software or faster delivery. The source cites Veracode testing in which 44% of AI coding tasks produced a known vulnerability; that result concerns the tested tasks and should not be read as a rate for all AI-assisted code. It nonetheless underlines why teams need to track security and rework alongside output.
The proposed measures are cost per accepted change, rework, change-failure rate and whether completed stories outpace incoming stories. These measures connect spending to work that is accepted and maintained, rather than to tokens consumed or code generated. They also give budget owners a way to compare uses of AI within a team, instead of assuming every task or developer benefits equally.
As an affiliate, we earn on qualifying purchases.
Evidence Varies by Team and Task
The results cited in the review range from no change in pull-request frequency to reported increases of several times at individual companies. One workplace saw pull-request counts stay flat while average pull-request size tripled; another reported a rise from roughly four to eight pull requests per developer per week. Such examples describe separate organizations, not a controlled comparison, and counts alone do not show whether changes improved products or reduced delivery time.
Controlled studies add uncertainty. METR’s early-2025 study found experienced open-source developers took 19% longer when using AI on the tasks examined. A February follow-up estimated an 18% speedup for returning developers and a 4% speedup for new developers, but both confidence intervals included zero. METR also said the data were unreliable because 30% to 50% of developers withheld tasks they did not want to attempt without AI. In a survey of 349 technical workers, the median respondent reported a 1.4-to-2-times change in work value; METR said participants in its 2025 work had overestimated AI’s effect by 40 percentage points.
Vendor data, company reports, surveys and controlled experiments answer different questions. They cover different users, tasks and time periods, so their results cannot be combined into a single dependable productivity multiplier. The review’s conclusion that individual output is probably up by double digits is the author’s interpretation, not a settled finding shared by all the cited research.
How Much Work Becomes Value
The evidence does not establish a reliable, general productivity gain or a standard return on investment. The 27-workplace interviews are not presented as a representative survey of all engineering organizations, while vendor figures describe activity on particular platforms. Company examples and self-reports can identify possible benefits but cannot isolate AI’s effect from other changes in staffing, workflow or project mix.
It is also unclear how widely reported gains persist once teams account for code review, security checks, defects and maintenance. The source provides no common cost-per-accepted-change results across the organizations, and it does not give a consistent baseline for comparing their spending. The scale of the reported vulnerability finding outside Veracode’s test tasks is unknown.
What Budget Owners Should Track
Companies will need to test AI use against their own work as they renew subscriptions and set budgets for 2027. The approach described in the review is to compare costs with accepted changes, rework, change failures and story completion, while recording the task type and the time period. Teams can then assess whether tools help with specific work, such as triage or backlog cleanup, without assuming the same result for complex legacy systems.
The next useful evidence would include repeated measurements before and after adoption, clear definitions of output, and the costs of review and defects—not just the number of prompts or pull requests. The source does not identify a shared industry standard or a forthcoming study that will settle the question. For now, company-specific outcome tracking remains the practical next step.
Key Questions
What is the main finding about AI coding tool value?
The evidence reviewed shows uneven results. Teams report useful gains in some tasks, but the cited studies do not establish one productivity increase that applies across companies.
Which tasks were reported to benefit?
The review points to issue triage, code archaeology, prototyping and backlog cleanup, particularly on simple and consistent technology stacks. These are reported examples, not a guarantee of similar results elsewhere.
Why are pull-request counts not enough?
More pull requests can reflect more output without showing whether the changes are useful, maintainable or safe. Microsoft study authors cautioned that a merged pull request does not equal delivered value.
What should companies measure instead?
The review recommends tracking cost per accepted change, rework, change-failure rates and completed stories compared with incoming stories. Those measures connect tool spending to delivery outcomes.
Does the evidence show AI coding tools are safe?
No general safety conclusion follows from the figures presented. The review cites Veracode testing in which 44% of AI coding tasks produced a known vulnerability, but the result applies to those tested tasks, not necessarily to all AI-assisted code.
Source: ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
