Longitudinal study finds AI productivity gains at 10%, not 10x
Preliminary data from a longitudinal AI impact study reveals that productivity gains from AI tools are more modest than often claimed. The study analyzed data from 40 companies between November 2024 and February 2026 to track whether teams ship more pull requests as AI adoption increases.
Key findings
During the study period, AI usage increased significantly—by an average of 65%. However, PR throughput only increased by 9.97%. This figure is particularly robust because researchers filtered out potential gamification effects by excluding teams that set PR throughput targets for individual engineers, which could drive metric inflation rather than genuine output.
What this means for engineering teams
The ~10% gain is consistent with what engineering leaders report more broadly: most organizations are landing in the 8–12% range. While this represents real improvement, it's far from the 2–3x gains many executives and boards have come to expect from AI adoption.
Why gains aren't higher
Developers across several organizations explained that writing code was never the bottleneck. As one senior developer noted: "The easy tasks are a little easier. The tedious tasks are a little less annoying. A four-day task might take three. But that doesn't mean I'm shipping 3x more PRs."
AI may accelerate the coding portion of the job, but coding represents a relatively small slice of how engineers actually spend their time. Planning, alignment, scoping, code review, and handoffs—the human parts of the SDLC—remain largely untouched by current AI tools.
Study methodology
The study is longitudinal, meaning it tracks changes over time rather than providing a single snapshot. The full study will explore why some teams capture more upside than others and what leaders can do to close that gap.
📖 Read the full source: HN AI Agents
👀 See Also
Go Players Disempower Themselves to AI: How Cheating Became Undetectable
The LessWrong post details how AI cheating in Go tournaments became rampant and nearly impossible to punish, using the case of Carlo Metta who used Leela 0.11 and Leela Zero to win 25 of 26 games over several seasons, with only one loss under camera surveillance.
Claude 4.6 Opus Can Reproduce Linux's list.h From Minimal Input
A user demonstrated that Claude 4.6 Opus can generate a near-identical copy of Linux's list.h header file when given the first 43 lines as input with temperature set to 0, raising questions about GPL licensing implications for AI models trained on open-source code.
EU Forces Google to Open Android AI to Third Parties Under DMA
European Commission proposes measures to allow third-party AI assistants system-level access on Android, including hot word invocation, screen context, and local model hardware access. Google calls it 'unwarranted intervention'.
Reddit Discussion Critiques Reactive AI Assistants, Calls for True Proactivity
A Reddit post argues that current AI assistants are reactive by design, waiting for human prompts rather than proactively identifying issues. The author distinguishes between scheduled checks and true contextual awareness, noting that real proactivity requires persistent memory, event-driven triggers, and cross-time reasoning.