Faster, or Just Busier? The Evidence on AI Coding Mandates

There's a particular kind of vertigo that comes from being told you're slow at something you're sure you're fast at. In 2025, a group of experienced open-source developers volunteered to find out which version of themselves was telling the truth.

The researchers at METR, a nonprofit that studies AI capabilities, recruited 16 seasoned engineers and had them work through 246 real tasks on codebases they knew intimately. The repositories averaged around 22,000 GitHub stars and more than a million lines of code, and the developers had spent an average of five years and roughly 1,500 commits living inside them. For each task, a coin flip decided whether the developer was allowed to use AI tools (mostly Cursor Pro paired with Claude 3.5 and 3.7 Sonnet) or had to work the old-fashioned way.

Before starting, the developers predicted AI would make them about 24% faster. Afterward, having actually done the work, they estimated they'd been roughly 20% faster with the tools. Independent machine-learning and economics experts had forecast speedups of 38–39%.

Every guess missed in the same direction. When these experienced developers used AI, they were 19% slower.

That result, published as "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity" (Becker, Rush, Barnes, and Rein; arXiv:2507.09089), is the finding senior engineers and engineering managers should keep closest at hand. Not because it proves AI is useless. It doesn't. But because it cleanly demonstrates a gap of nearly 40 percentage points between how fast skilled developers felt and how fast they actually were. The tools were genuinely pleasant to use; about 69% of the developers kept using them after the study ended. They just weren't faster.

That gap explains much of the 2025–2026 evidence: the conditional productivity gains, the review bottleneck, the mixed burnout story, the skill risks, and the growing wave of companies deciding to force adoption before the data supports that much confidence.

The productivity story is real, smaller, and more conditional than the sales pitch#

Why the experts got slower#

When METR went back through 143 hours of screen recordings, about 29% of the total time logged, the mechanism became clear. With AI in the loop, time drained away from active coding and searching and pooled instead in prompting the model, waiting for it to respond, and reviewing what it produced. There was also simply more idle time. Over half of the AI's suggestions were judged unusable, and even the accepted ones often needed meaningful correction. Most tellingly, the AI was least useful exactly where these developers were strongest: deep inside a codebase they already understood better than any model could.

A senior engineer's edge is context: knowing why a weird workaround exists, which module is fragile, what the implicit conventions are. An AI assistant can't share that context, so on familiar, complex code it mostly generates plausible-looking suggestions that the expert then has to read, evaluate, and fix. The reviewing tax exceeds the typing savings.

An honest caveat that cuts the other way#

It would be dishonest to present the 19% figure as a law of nature, and METR itself was the first to push back. The study was 16 developers with a wide confidence interval (the real effect could plausibly have been anywhere from 2% to 39% slower). And in February 2026, METR published an update describing a larger follow-up it had launched in August 2025: 57 developers, more than 800 tasks, using newer agentic tools like Claude Code and Codex. That study hit a wall: between 30% and 50% of invited developers refused to work without AI at all, which biased the sample so badly the signal became unreliable. The newer cohort showed only about a 4% slowdown (confidence interval −15% to +9%), and METR's tentative read was that "AI likely provides productivity benefits in early 2026."

Hold the METR result carefully: AI is not a universal speedup, and for experts working on mature, familiar codebases it can be a net slowdown. It is not "AI always slows everyone down." The conditions matter enormously, which is the through-line of every good study in this space.

The conditional map: where AI helps and where it hurts#

The most useful single picture of those conditions comes from Yegor Denisov-Blanch's Stanford research group, presented at the AI Engineer World's Fair in 2025. Their dataset is enormous, roughly 100,000 engineers across more than 600 companies and billions of lines of private code, and their headline is deliberately unglamorous: a net average productivity gain of about 15–20% once you account for rework, with staggering variance underneath that average.

Read that map next to the METR result and the apparent contradiction dissolves. Both are saying the same thing: AI's value is highest for the unfamiliar, the simple, and the new, and lowest for the complex, the legacy, and the deeply-known. Seniority correlates strongly with the second category.

The seniority gradient, confirmed from several directions#

This isn't just inference. The pattern that less-experienced developers benefit more shows up repeatedly. GitHub's early, widely-cited randomized controlled trials set the optimistic tone: Peng and colleagues found developers completing a specific task 55.8% faster, and a larger experiment (Cui and colleagues, run with Microsoft, MIT, Princeton, and Wharton across 4,867 developers at Microsoft, Accenture, and an anonymous Fortune 100 firm) found roughly 26% more pull requests per week. But even in that cheerful data, the gains concentrated among the less experienced. The same field experiment found developers with above-median tenure showed no statistically significant productivity increase at all. GitHub's own telemetry shows juniors accepting more suggestions and getting more out of them.

The intuition is almost paradoxical: senior engineers benefit least because they're skilled enough to notice when the AI is wrong. Noticing, evaluating, and correcting is precisely the overhead that erases the speed gain.

What happens to the code itself#

Speed is only half of productivity. The other half is whether the code you produce is any good, and here the trend lines are concerning. GitClear's 2025 "AI Copilot Code Quality" report (by Harding and Kloster) analyzed 211 million changed lines of code from 2020 through 2024, including repositories from Google, Microsoft, and Meta. The findings describe a codebase quietly accumulating maintenance weight:

• An 8x increase in duplicated code blocks in 2024.
• For the first time in the history of their data, copy-pasted code surpassed refactored ("moved") code.
• Refactoring's share of all changes fell from about 24% in 2021 toward a projected ~3% in 2025.
• Short-term churn (code revised or reverted within two weeks of being written) rose from 3.1% in 2020 toward a projected 7%+.

The interpretation is straightforward: AI assistants optimize for fast insertion (hit Tab, accept the suggestion) over thoughtful reuse and refactoring. Every duplicated block is future maintenance cost and defect risk; the research GitClear cites links duplicated code to 15–50% more defects.

The metric that should keep managers up at night: throughput up, stability down#

This is where individual productivity and team health start to diverge.

Google's DORA program, the gold standard for measuring software delivery performance, found in its 2024 report that every 25-percentage-point increase in AI adoption was associated with a 7.2% decrease in delivery stability and a small drop in throughput, even as individuals reported feeling more productive. (There was a small ~3.4% improvement in code quality at the individual-block level, so it's not all bad.) The 2025 DORA report, "State of AI-assisted Software Development" (Google Cloud, nearly 5,000 respondents, led by Nathen Harvey), found adoption had gone nearly universal: per the report, AI use among software professionals surged to roughly 90% (a 14% jump in a year), with developers spending a median of about two hours a day working with it.

In 2025 the throughput relationship flipped positive. But the instability association persisted, and DORA explicitly tested and rejected a comforting "fail fast, fix fast" hypothesis: the data didn't support the idea that teams were simply breaking things faster and recovering faster. DORA's framing has become the most quoted line in the field: AI is an amplifier. It doesn't fix a struggling team; it intensifies whatever is already there. Strong teams get stronger; weak teams get their problems magnified. DORA also documented a "trust paradox": more than 80% of developers feel more productive and 59% report code-quality benefits, yet about 30% report little or no trust in the AI-generated code they're shipping.

The review bottleneck, quantified#

If you want to know where all that instability is coming from, follow the pull requests. Faros AI's telemetry (10,000+ developers across 1,255 teams, June 2025) named the phenomenon the "AI Productivity Paradox." Teams with high AI adoption completed 21% more tasks and merged 98% more PRs, and yet their organizational DORA metrics (lead time, deployment frequency, change failure rate, mean time to restore) stayed flat. The reason showed up downstream: PR size grew about 154%, review time rose 91%, and bugs ticked up 9%. By Faros's 2026 dataset (now 22,000 developers), the trend had escalated into what they called "Acceleration Whiplash": median time spent in PR review up 441%, 31% more PRs merging with no review at all, PR size up 51.3%, bugs per developer up 54%, and incidents per PR up a startling 242.7%.

The pattern is predictable, since AI accelerates the writing of code, but writing was never the bottleneck. Review was. Pour more, larger, faster-produced code into a review step that was already constrained, and the whole system jams. Even Cursor's CEO Michael Truell conceded that reviewing code "looks the same as it did three years ago," and Cursor promptly went and acquired the code-review startup Graphite.

Well-being and burnout, where the obvious story is partly wrong#

The intuitive narrative says AI tools and AI pressure are torching developer morale. The evidence is more mixed.

The counterintuitive null result#

DORA's 2025 report found that AI has no measurable effect on burnout or friction. Those, the researchers concluded, "remain properties of the organizational system rather than the individual workstation." Harvey put it bluntly: if an engineer is burned out, throwing a new tool at them won't help. This echoes DORA's 2024 finding that the real burnout engine is unstable organizational priorities (the constant reshuffling of goals and direction), and that this kind of burnout is highly resistant to mitigation. In other words: management churn burns people out. Copilot, by itself, mostly doesn't.

That should temper the alarmism. But it is not the whole picture.

Burnout is high, and it's worst at the top of the org chart#

Even if AI isn't the cause, the surrounding environment is genuinely strained. Jellyfish's 2025 State of Engineering Management Report (around 640 respondents) found 90% of teams now using AI coding tools (up from 61%) and 62% reporting productivity gains of 25% or more, yet 46% expect engineer burnout to increase, against just 21% expecting it to fall. Jellyfish's 2024 baseline is starker still: 65% of all respondents had experienced burnout in the prior year, a figure that climbed to 85% of managers and 92% of executives at companies with 500+ engineers. LeadDev's 2025 Engineering Leadership Report (617 leaders) found 22% at critical burnout levels and only 21% describing themselves as "healthy," with 40% saying their teams are less motivated than a year ago, 65% carrying expanded responsibilities, and 38% working longer hours. And Stack Overflow's 2025 Developer Survey (49,000+ respondents) recorded positive sentiment toward AI tools falling from above 70% in 2023–24 to 60%, with experienced developers the most skeptical group, posting the highest "highly distrust" rate, around 20%.

"Babysitting the AI": the real mechanism for seniors#

If there's a single well-being mechanism that hits senior engineers hardest, it's the quiet transformation of their job from author to reviewer. Harness's "State of Engineering Excellence 2026" survey (700 practitioners and managers) found 81% of leaders saying developers now spend more time reviewing AI-generated code, with roughly a third of developer time swallowed by "invisible work": reviewing AI code (53%), fixing subtle AI-introduced bugs (52%), explaining AI-generated code to teammates (48%), and context-switching (45%). A separate Harness survey of 500 leaders (reported via DevOps.com) found 67% spending more time debugging AI code, 68% more time on AI-introduced security issues, and 92% saying AI increases the "blast radius" of bad code that needs debugging.

There's a cognitive reason this is so draining. Reading code is harder than writing it, because the reviewer has to reconstruct the author's intent, and an AI has no intent to reconstruct. You're reverse-engineering the reasoning of something that wasn't reasoning. Do that for a third of your day and the fatigue is real, even if no survey instrument files it under "AI-caused burnout."

Managers, caught in the middle#

Engineering managers occupy the worst seat. They're under pressure to demonstrate AI ROI that, at the enterprise level, mostly hasn't materialized. McKinsey's "The State of AI" (November 2025; surveyed late June through late July 2025; 1,993 respondents across 105 nations) found that while 88% of organizations now use AI, just 39% report any EBIT impact at the enterprise level, and only about 6% (the so-called "AI high performers") attribute 5% or more of EBIT to it. Jellyfish found 61% of companies increased engineering budgets in 2025 but only 20% measure AI's impact with actual engineering metrics. As one CIO-focused analysis of the Harness data put it, leaders are being asked to make multi-year AI investment decisions using dashboards built for a different era.

Meanwhile the distance between leadership and the trenches is widening. Atlassian's 2025 State of Developer Experience report (3,500 developers and managers) found a growing "empathy gap": 63% of developers feel their leaders don't understand their pain points, up from 44%. Part of the reason is that leaders bank the AI time-savings on a spreadsheet without ever confronting the friction those savings created downstream.

"Workslop" and the social cost of low-effort AI output#

A particularly sharp study here comes from BetterUp Labs and the Stanford Social Media Lab (Niederhoffer, Hancock, and Rapuano; about 1,150 U.S. desk workers, September 2025, published in Harvard Business Review). They coined the term "workslop": AI output that looks polished but is hollow, passing the work of actually thinking onto the recipient. Forty percent of workers had received workslop in the prior month, and managers were more exposed than individual contributors (54% versus 38.5%). Each incident took an average of nearly two hours to untangle, costing roughly $186 per employee per month. But the real damage was social: people who received workslop rated the senders as less capable, less trustworthy, and less intelligent. The researchers tied the phenomenon directly to "indiscriminate AI mandates," which brings us to a recurring villain.

Job-security anxiety is not paranoia#

The fear that AI productivity gains will be cashed out as headcount cuts is grounded in real events. Coinbase cut roughly 700 employees (about 14% of its workforce as of May 1, 2026), and its May 5, 2026 SEC Form 8-K described the plan as optimizing operations "for the AI era." CEO Brian Armstrong said he wanted to rebuild Coinbase "as an intelligence, with humans around the edge," targeting "no pure managers" and a maximum of five organizational layers (reported by CNBC and Fortune, May 5, 2026). Salesforce's Marc Benioff, on the company's Q4 FY2025 earnings call (February 26, 2025), said plainly that Salesforce would not hire any new engineers that year, citing a 30% engineering productivity increase it intended to "ride."

That backdrop makes surveillance fears combustible. Harness found 54% of practitioners afraid of performance evaluations based on their AI-usage data, and managers roughly four times more likely than developers to report no concern about that surveillance (15% versus 4%). That perception gap is corrosive: the people being watched are far more worried than the people doing the watching realize.

Focus and the texture of the workday#

The cognitive-focus picture is mixed.

On one side, AI can make coding feel lighter. METR's screen recordings showed more idle time during AI-assisted work, and analysts like Sean Goedecke have argued that AI-assisted coding simply demands less moment-to-moment cognitive effort, which makes multitasking or even gently zoning out possible. That feels pleasant (recall those 69% of METR developers who kept using the tools), but it can quietly mask reduced engagement with the problem.

On the other side, the tools fragment attention in concrete ways. Atlassian found context-switching between tools to be a top time-waster, and that developers lose 10+ hours a week to non-coding friction even while saving roughly 10 hours through AI, close to a wash. Telemetry from DX and similar platforms warns of "spiky AI commits" that signal disruptive context switching. And the constant evaluation of output that's almost right (the single most-cited developer frustration in Stack Overflow's 2025 survey, at 66%) is a sustained, low-grade cognitive tax that never quite lets you settle into flow.

The fairest synthesis: AI can usefully shift cognitive load away from rote syntax recall and toward higher-level judgment, which is a real benefit. But the prompt → wait → review loop chops up the deep-focus states that complex engineering actually requires.

The long game: learning, skill atrophy, and the mentorship pipeline#

This is where the short-term-speed-versus-long-term-capability tension lives. For senior engineers and managers, the stakes are unusually high: you are both the people most at risk of quiet deskilling and the people responsible for producing the next generation of seniors.

Cognitive offloading and the erosion of judgment#

A Microsoft and Carnegie Mellon study (Lee and colleagues, CHI 2025; 319 knowledge workers) found that higher trust in AI correlated with less critical thinking. Their warning is worth sitting with: automating routine tasks can "deprive the user of the routine opportunities to practice their judgement," leaving those faculties "atrophied and unprepared when the exceptions do arise." They also found AI users produced a less diverse set of outcomes: everyone converging on the same model-shaped answer. A separate study by Gerlich (2025; 666 participants) found frequent AI reliance negatively associated with critical-thinking skills via cognitive offloading, an effect strongest among younger users.

"Cognitive debt": the brain-on-AI study#

The most provocative entry is the MIT Media Lab's "Your Brain on ChatGPT" (Kosmyna and colleagues; arXiv:2506.08872, June 2025), which used 32-electrode EEG on 54 participants writing essays in three groups: LLM-assisted, search-engine-assisted, and brain-only. LLM users showed the weakest neural connectivity, the lowest sense of ownership over their own work, and, strikingly, struggled to quote from essays they had just "written." Over four months they consistently underperformed at the neural, linguistic, and behavioral levels. When LLM users were later switched to working brain-only, they showed weaker engagement than the group that had been brain-only all along, suggesting an accumulated "cognitive debt" that doesn't immediately clear when you put the tool down.

The caveats here are serious and you should weight them: the sample was small (only 18 participants completed the fourth session), it studied essay-writing rather than coding, it used a single model (GPT-4o), and a 2026 published critique (arXiv:2601.00856) raised methodological concerns about sample size, reproducibility, and the EEG analysis. Treat it as suggestive of a real risk, not as proof.

The mentorship pipeline is the part that actually breaks#

For managers, the most consequential long-term effect may not be individual skill atrophy at all. It may be the slow collapse of the apprenticeship that turns juniors into seniors. LeadDev's AI Impact Report 2025 found 38% of leaders agreeing that AI has reduced the direct mentoring juniors receive from seniors, and 54% believing AI will reduce junior hiring outright. The traditional learning loop (a junior struggles, debugs, gets stuck, asks a senior, and internalizes the lesson) gets short-circuited when the junior simply prompts an AI and receives working code without ever passing through the productive struggle that builds durable mental models.

A UC Berkeley and Yale study (published in HBR, February 2026; 200 employees tracked over 8 months) found seniors now absorbing work that previously required several people, with rising workloads and review burdens, while juniors grow more isolated. Stack Overflow's 2025 survey reinforced the isolation theme: only 17% of agent users said agents improved team collaboration, the lowest-rated impact of all. Microsoft's Mark Russinovich and Scott Hanselman, in "Redefining the Software Engineering Profession for AI", propose borrowing a "preceptorship" model from nursing (deliberately carving out 20–30% of senior engineers' time for structured mentorship) precisely because AI is eroding the path from junior to senior. AWS CEO Matt Garman was blunter, calling the idea of replacing junior engineers with AI "one of the dumbest things I've ever heard," and asking how it's supposed to work when, ten years out, you have no one who ever learned anything.

A telling signal from the tools themselves#

Anthropic's own Economic Index reports (2025–2026) add a quietly important data point. Coding is Claude's single dominant use, and Claude Code skews heavily toward automation (79% of conversations) rather than augmentation (21%), compared with a much more balanced 49% automation on the consumer Claude.ai. In other words, the agentic coding tool nudges users toward fully delegating tasks rather than collaborating on them. Across Anthropic's platform, automation briefly overtook augmentation in August 2025 (directive use rose from 27% to 39%) before augmentation edged back ahead by November. Anthropic's data also shows that higher-tenure, higher-skill users get more successful responses, hinting at a skill-biased divide where the tools reward the people who least need them.

The mandates: when "you should try this" becomes "use it or you're fired"#

Layered on top of every effect above is a distinctly 2025–2026 phenomenon: companies no longer suggesting AI but requiring it, often tying it to performance reviews, hiring, and layoffs. The documented cases are not subtle.

Shopify#

On April 7, 2025, CEO Tobi Lütke published an internal memo (he posted it publicly because it was already leaking) declaring that "reflexive AI usage is now a baseline expectation at Shopify." AI competency was added to performance and peer reviews. Teams now have to justify why a task can't be done by AI before they're granted additional headcount. "Stagnation," Lütke wrote, "is slow-motion failure." Over the following year Shopify's headcount drifted from roughly 8,300 to 8,100.

Microsoft#

In June 2025, Julia Liuson sent managers an internal memo stating that using AI is "no longer optional" and "core to every role and every level," and instructing them to factor AI use into "holistic reflections" on performance, with some teams weighing formal metrics. GitHub CEO Thomas Dohmke called the practice "totally fair game." The push was partly aimed at lagging internal Copilot adoption as Microsoft watched competitors like Cursor surge.

Coinbase#

Brian Armstrong gave engineers one week to onboard Copilot and Cursor, then held a Saturday meeting for the holdouts and fired those without a good reason. He later described it as "I went rogue." Coinbase now reports roughly a third of its code is AI-written, with a 50% target, mandatory human review, and monthly "AI speed runs." Armstrong acknowledged that "some people really didn't like" the heavy-handed approach. (The same logic visibly drove the 14% workforce cut described earlier.) Notably, even Armstrong drew a hard line: no "vibe coding" systems that move money.

Duolingo#

CEO Luis von Ahn's April 2025 "AI-first" memo told the company to stop using contractors for work AI could do, folded AI into hiring and review decisions, and touted 148 AI-built courses. It triggered a fierce social-media backlash, and Duolingo walked it back. Von Ahn later said plainly, "we backtracked," reframing performance as simply doing the job well and stating that no full-time employees would be laid off because of AI. It's the clearest cautionary tale in the set.

The shorter cases#

Thomson Reuters warned that staff who don't adopt AI "will be left behind." IBM paused hiring for roughly 7,800 back-office roles it believed AI could replace. Salesforce, as noted, declared no new engineers in 2025. KPMG folded AI usage into its 2026 reviews.

Do mandates actually work?#

They reliably move adoption metrics. Whether they move value is a different question, and the evidence leans skeptical. WalkMe (an SAP company), in its fifth annual State of Digital Adoption report (released April 9, 2026; 3,750 respondents across 14 countries at enterprises of 1,000+ employees), found that 54% of workers had bypassed AI tools and done a task manually at least once in the prior 30 days, a further 33% hadn't used AI at all, and only 9% of workers trusted AI for complex, business-critical decisions, against 61% of executives. That 52-point trust chasm between the people mandating AI and the people using it is the whole story in one number.

The constructive counter-evidence comes, again, from DORA: a clear, well-communicated organizational stance on AI reduces friction and amplifies AI's positive effects. The lever that works is policy clarity and genuine enablement, not coercion. Forrester's analysis of the Shopify memo specifically warned against "reflexive" use, noting that the most AI-savvy employees are precisely the ones who know when not to reach for it. And the metric-gaming risk is well understood in the research literature (Thomas and Uminsky, "The Problem with Metrics Is a Fundamental Problem for AI"): counting tokens, sessions, or lines of code is trivially gameable and only weakly correlated with anything that matters.

Which leaves senior engineers and managers structurally squeezed: pressed from above to adopt fast and show ROI, while standing on ground where the honest data says AI helps least (complex, legacy systems), the review burden balloons, and the quality risks are real enough that even the executives mandating AI won't let it touch the code that moves money.

So what should you actually do?#

The evidence doesn't say "refuse AI" and it doesn't say "mandate it everywhere." It points to a more disciplined approach.

If you manage engineers#

Measure before you mandate. Establish a pre-AI baseline across the four DORA metrics (deployment frequency, lead time, change failure rate, time to restore) plus review time, PR size, rework and churn, and incidents per PR. Both Faros and METR show individual output and team delivery pulling in opposite directions, so if you track only commits and merged PRs you'll mistake activity for value. The trigger to act: if PR size, review time, or change-failure rate climb materially after rollout, you have an Acceleration Whiplash problem.

Wall off improvement data from performance evaluation. Fifty-four percent of developers fear being evaluated on AI-usage data, and most want it kept separate. Publicly commit to not ranking individuals on token or session counts. It's the single cheapest move you can make to preserve trust.

Point AI at friction, not at coding. Coding is only about 16% of a developer's week (Atlassian) and was never the real bottleneck. Information discovery, testing, context-switching, and documentation are. Aim AI there for genuine well-being gains.

Defend the mentorship pipeline like it's load-bearing, because it is. Use review-first workflows where juniors critique AI output, schedule periodic no-AI learning exercises, and seriously consider the Microsoft preceptorship model of protected senior mentoring time. Track whether your juniors can still independently debug a production incident. That's your early-warning system.

Set quality guardrails. Mandatory human review of AI code (non-negotiable in regulated, financial, or security-sensitive paths), duplication and churn monitoring in the GitClear style, and a "trust but verify" culture. Tighten the moment your duplicate-block rate or short-term churn climbs above baseline.

If you're a senior developer#

Calibrate against your own optimism. Assume your felt speedup overstates the real one. That's the lesson of METR's 39-point gap. Reach for AI on greenfield work, unfamiliar languages, boilerplate, and exploration; be wary of it on the complex, familiar, legacy code where you're already the expert and where it tends to cost you.

Guard your own skills deliberately. The cognitive-debt and atrophy research is an argument for keeping core reasoning and debugging muscles in regular use. Use it or lose it applies to you too.

Own the review constraint. Push your org to treat code review (not code generation) as the real bottleneck, and to invest in smaller PRs, better review tooling, and AI-assisted review, rather than accelerating generation straight into a downstream traffic jam.

What would change this advice#

If METR's redesigned 2026+ study and the newer agentic tools (Claude Code, Codex, Opus-class models) start showing consistent, statistically robust speedups for experts on mature codebases, and if DORA-style stability metrics stop deteriorating at high adoption, the case for broader, faster, less-guarded adoption gets much stronger. The things to watch are the next METR uplift report, DORA 2026, and independent, non-vendor replications of the Stanford complexity map.

How to read all of this without fooling yourself#

A few caveats matter because this field is noisy, and almost everyone selling you a conclusion is also selling you something else.

• The headline numbers have wide error bars. METR's 19% came from 16 developers with a confidence interval spanning 2% to 39%, and METR itself later revised its design and tentatively concluded AI probably does help in early 2026. Hold the 19% as "not a universal speedup for experts on mature code," not as gospel.
• Watch for vendor bias. Jellyfish, Harness, Faros, Atlassian, GitHub, Anthropic, and DORA's sponsors all sell into this market, and their framing tends to land on "AI works, but you need our product to do it right." The independent academic work (METR, MIT, CMU, Stanford) and the cross-vendor convergence on roughly 10% system-level gains are more trustworthy than any single vendor's number.
• The tools keep changing underneath the studies. GitClear's data ends in 2024; the famous GitHub RCTs used GPT-3.5; METR used Claude 3.5 and 3.7. Agentic tools and late-2025 models shifted the landscape, so every conclusion here has a short shelf life.
• Don't conflate the burnout findings. DORA found no measurable AI effect on burnout: organizational factors dominate. Jellyfish's 46% and LeadDev's 22% measure burnout expectations and general burnout, not AI-caused burnout. The honest synthesis: AI isn't proven to cause burnout directly, but mandate-driven pressure, surveillance fear, review overload, and ROI expectations are all plausible aggravators.
• The MIT EEG study is preliminary. Small sample, essay-writing rather than coding, one model, and a published methodological critique. Suggestive, not settled.
• Correlation isn't causation. GitClear's quality decline coincides with AI adoption, but also with a wave of tech layoffs. DORA's instability link is associational. And effect sizes swing wildly with context: codebase maturity, language, task complexity, team practices. That context-dependence is, in the end, the single most robust finding in the entire literature. The right question is never "does AI make developers faster?" It's "faster at what, for whom, on which code, and at what cost downstream?"