TL;DR
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A report by The Pragmatic Engineer gathered AI spending and pull-request data from 27 workplaces, alongside platform data from CodeRabbit and Gitar. CodeRabbit reported roughly doubled PR volumes among surveyed users over a year, but the report found companies generally could not say whether higher AI spending produced measurable financial returns.
A report from The Pragmatic Engineer found that software companies are spending widely varying amounts on AI tools, while many still cannot quantify the return. Drawing on conversations with engineering leaders and senior engineers at 27 workplaces, plus aggregate data from CodeRabbit and Gitar, the report describes increases in pull-request activity but does not establish that AI spending caused those increases or improved business results.
CodeRabbit shared anonymized platform data showing a 113% increase in weekly pull requests among surveyed data engineers and a 127% rise in monthly PRs. In a separate comparison tracking the same group of 20,000 developers for a year, the company reported a 128% increase. These are platform observations, not a controlled test of AI’s effect: the report does not provide evidence that isolates AI tool use from other changes in teams or work practices.
Gitar, an AI code verification agent, supplied data on PRs per developer across hundreds of customer teams. Its data placed some “AI-native” teams above 60 PRs per developer per week. The report’s author cautioned that company norms and culture may help explain different output levels, and PR volume alone does not show code quality, time saved, or value delivered to customers.
Spending varied sharply in the company accounts. A two-person .NET shop reported spending $3,000 per developer per month, while a publicly traded company with 1,500 engineers capped monthly spend at $150 per developer. A trading firm described in the report said its total AI spend reached $8 million a month in September, up from $500,000 in January. Those figures are examples from the surveyed organizations, not estimates of typical industry spending.
Why PR Counts Fall Short of ROI
The findings show why engineering leaders are looking for a practical way to assess AI budgets: spending is rising, but readily available measures such as PR counts capture activity, not necessarily productivity or financial value. More PRs could reflect faster delivery, smaller changes, different workflows, or simply a change in how teams divide work. The report does not determine which explanation applies across the companies surveyed.
That distinction matters for decisions about renewals, budgets and employee access. Several companies described spending caps, while some said subsidized subscriptions were nearing their end. The report says some organizations anticipated that bills could rise five to 10 times if subsidies ended and costs were not optimized. Leaders therefore face a measurement problem as well as a cost question: whether additional spending produces benefits that justify its price.
For engineers, the evidence also cautions against treating output volume as a full measure of performance. A rise in PRs does not, on its own, establish that software is safer, easier to maintain or delivered sooner to users. The report presents useful indicators of changing activity and spending, but not a settled business case for AI coding tools.
As an affiliate, we earn on qualifying purchases.
How the Spending Data Was Collected
The Pragmatic Engineer author said an engineer asked how much companies spend on AI per developer and whether PR throughput changed before and after that spending. The report draws on interviews with engineering leaders and senior-plus engineers at 27 workplaces, with most company identities withheld at their request. It also includes aggregate data supplied by CodeRabbit and Gitar.
The data comes from a limited set of workplaces and vendor platforms, rather than a representative industry-wide survey. The report describes PRs as a readily available measure that can be compared over time, but it does not present PR counts as a complete measure of engineering productivity. It also notes that spending can be hard to track when companies subsidize subscriptions or pay through API plans.
Across the accounts, the report says company size did not correspond neatly with spending per developer. It also describes spending caps as common among medium-and-higher spenders, and says some organizations’ monthly costs had fallen from summer peaks, in part as they moved to open models. The findings are a snapshot of the companies interviewed and the platform data shared, not a market-wide accounting.
“Lots of traders are using Claude. Engineering is learning the importance of rate limiting for traders who accidentally run expensive queries without realizing it.”
— An engineer at the trading firm, as quoted in The Pragmatic Engineer
AI-powered pull request management software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What the Output Data Cannot Show
The report does not establish that AI tools caused the reported increases in PR activity. It also does not provide a consistent before-and-after financial analysis linking tool costs to revenue, savings, delivery time or software quality. Whether higher PR counts translate into measurable business gains remains unclear.
The survey is limited to 27 workplaces, mostly anonymized, and the vendor metrics cover users or customers of particular platforms. The report does not describe a representative sampling method or a common definition of AI spending across all companies. Its examples should not be read as industry averages. The timing and comparability of individual companies’ spending accounts also vary.
It is also unclear how much reported spending reflects engineers alone. One firm said traders were its largest AI spenders, and the report notes that subscriptions can obscure per-user costs. The available data therefore leaves open both who is driving bills and how much of the measured output change is attributable to AI.
As an affiliate, we earn on qualifying purchases.
How Companies May Track Returns
The report does not announce a shared standard for measuring AI ROI or identify a forthcoming study that will settle the question. The next step for companies, based on the issues it documents, is to compare tool costs with outcomes that go beyond PR volume, while accounting for changes in team size, work mix and software quality. Those are measurement challenges, not results established by this report.
Organizations facing the end of subsidized subscriptions may need to decide whether to limit usage, negotiate or pay for access, or shift workloads to lower-cost models. The report says some companies already use spending caps and that several are preparing for possible cost increases. Whether those controls can preserve any productivity gains remains unknown.
As an affiliate, we earn on qualifying purchases.
Key Questions
Did the report prove AI tools increase software engineers’ productivity?
No. It reports higher PR counts in CodeRabbit platform data and high output at some Gitar customer teams, but does not show that AI caused the changes or establish gains in quality, time saved or business value.
How many workplaces were included?
The Pragmatic Engineer author said they spoke with engineering leaders and senior engineers at 27 workplaces. The report also used aggregate data provided by CodeRabbit and Gitar.
How much do companies spend on AI per engineer?
The examples varied substantially: one two-person .NET shop reported $3,000 per developer per month, while a large public company capped spend at $150. These are individual examples, not an industry average.
What does a higher number of pull requests mean?
It indicates more PR activity in the measured group, but does not by itself show that more valuable software was delivered or that work became faster or better. Team norms and workflow changes may also affect the count.
Can companies calculate AI’s ROI from this report?
No. The report says few companies can answer the ROI question so far. It presents spending accounts and output measures, but does not provide a consistent financial comparison showing benefits against AI costs.
Source: rss
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
