Will AI replace software engineers?
The only randomised controlled trial on this measured the opposite of what everyone predicted, including the developers in the experiment.
Key figures
Experienced developers using AI tools took 19% longer to finish real tasks in their own repositories.
The same developers believed AI had made them 20% faster.
Meanwhile employment of software developers aged 22 to 25 is down nearly 20% from its late-2022 peak.
The one randomised trial
Most claims about AI and developer productivity are vendor benchmarks or surveys of how people feel. METR ran an actual randomised controlled trial: 16 experienced open-source developers, 246 real tasks, in repositories they had worked on for an average of five years. Tasks were randomly assigned to allow AI tools or not.
When AI was allowed, completion time went up by 19%.
That is the opposite of the expected result, and it is worth being precise about what it does and does not say. These were experts working on code they already knew deeply. It is not a finding about a junior developer on an unfamiliar codebase, which is a different job.
Everybody guessed wrong
Economists predicted a 39% speed-up. Machine learning experts predicted 38%. The developers themselves predicted 24% before starting. After finishing the tasks, having actually been slowed down, they still estimated AI had made them 20% faster.
That last number is the one to sit with. The slowdown was invisible from the inside. People who had just lived through the experiment could not feel the direction of the effect, let alone its size.
Why it can slow an expert down
The mechanism is not mysterious once stated. An expert on a familiar codebase already knows what to write. Using AI adds prompting, reading a suggestion, judging whether it fits conventions the model cannot see, and repairing it when it nearly fits. Each step is small. Together they can exceed the time saved by not typing.
Nearly fitting is the expensive case. Code that is obviously wrong is discarded in seconds. Code that is plausible has to be read properly, and reading someone else's plausible code is slower than writing your own.
The hiring number points the other way
Stanford's payroll analysis found employment of software developers aged 22 to 25 down nearly 20% from its peak in late 2022. Software development is one of the occupations it identifies as most exposed, alongside customer service representatives, accountants and receptionists.
So two solid pieces of evidence point in opposite directions: the tools did not make experienced developers faster, and firms have cut junior hiring anyway. Both can be true. Hiring decisions are made on expectations, and the expectations were the 39% and 38% numbers in the chart above, not the measured result.
So will it replace them
On the evidence available: not the experienced ones, not yet, and not because the tooling has been shown to do their job faster. What is measurably happening is narrower and more specific. The rung people used to climb onto is being pulled up, in a profession where the standard route to expertise was doing junior work badly for a couple of years until you did it well.
The open question is not whether AI can write code. It plainly can. It is where the next cohort of people who can review that code is supposed to come from.
Sources and method
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, Becker, Rush, Barnes and Rein, METR, submitted 12 July 2025. Source of the 16 developers, 246 tasks, the 19% slowdown and all four forecast figures.
- Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence, Brynjolfsson, Chandar and Chen, Stanford Digital Economy Lab. Source of the nearly 20% employment fall for software developers aged 22 to 25 from the late-2022 peak.
- TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks, Carnegie Mellon University and collaborators. On a benchmark of 175 consequential office tasks the strongest agent completed 30.3% autonomously, which is the wider context for claims about autonomous software work.
- The METR trial used early-2025 tooling. It does not measure later model versions, and its participants were experts in repositories they knew well.
- Every source here was opened and checked on 25 September 2026. Where a question has no measurement behind it, this page says so rather than dressing an opinion as data.