AI coding agents generate more code, but not more software (arstechnica.com)

> I think senior engineers find it very difficult to give up on line-by-line review

i am pretty senior. I hate code reviews so much. They are so boring, easily the most tedious part of my job. Its such a chore. Who is clinging to line-by-line review. Ppl dont really give that much about their employers.

I think non-senior engineers don’t realize just how much of a burden is barely reviewed ai generated code.

I’ve seen a few projects that heavily rely on ai with not very good (though senior) devs driving it - and its a mess. Not the code. The resulting product. Code is actually sound as ai is pretty good now.

All because ai does code reviews and dump decisions go into prod

I agree. The latest models are getting quite good at validation, as long as they are clear on what the requirements are. Recently I got frustrated that while we have a huge test suite I noticed the LLM was hardly ever breaking a test with its changes (PRs only added tests, never changed old ones) which seemed suspicious.

I had it do a full audit. There were about 100k lines of tests, of which it determined about 60k were near useless. Many were overly mocked, some mocked to the point that they really weren't testing anything at all. It deleted those and wrote more expansive scenario tests to replace them, caught a lot of bugs on the way and fixed those too. Total time was about 2 hours.

I think humans need to review requirements, specifications, and test scenarios. And for some industries and uses they're still going to have to review every detail. But for a typical CRUD app, I think we're at a point where we can already set the process and quality bar higher than ever, in addition to moving faster.

We shouldn't give up on line-by-line review for serious applications.

My company's leadership is oscillating every few weeks on AI goals. Their most recent story is "we're expecting developer velocity to approximately double over the new 2-3 years as people get better at using AI". But of course that's completely non sequitur. People on my team spend about 10% of their time writing code and 90% doing design and analysis. So if AI doubles our code writing velocity, then it will make us 5% faster overall.

You have to classify each code review based on what it changes and decide how to handle it, agent review and/or human review. You can have an agent do that.

You don't give up on line-by-line review just because the human is getting better either, and senior engineers want other people to do line by line review on their own code. It's not a matter of how good or bad the person or program generating the code is.

My experience with prototyping games is that you need very strong human driven guidance and review in the beginning, then you can increasingly take your hands off the wheel.

If you take it off too early you end up with shit and will spend weeks fixing all these baked bad assumptions your project is built on top of. I spent weeks fixing some of this shit then the LLM revived a not insignificant portion of it in a later change because it got "confused" (its words) because some of the comments it previously made were left around.

> I think senior engineers find it very difficult to give up on line-by-line review, no matter how much better the agents have gotten.

I actually dislike traditional code review. In part this is because most of our code review tools are very cumbersome to use (GitHub's and GitLab's web interfaces in particular).

But also because humans aren't all that great at code review when it comes to correctness and security. Part of the problem is that you don't have access to the author's mental model for non-trivial changes and have to reconstruct it, otherwise your understanding of the program logic will be inadequate.

The problem is that purely agentic code review has orthogonal problems, and typically remains subpar when it comes to judging architecture and design considerations, compared to human reviewers. Both false positives and false negatives also remain a concern.

My best results so far have been with agent-assisted code review in a proper IDE, helping you to quickly construct a mental model and to analyze the underlying logic (in addition to having agents do a first pass for correctness issues).

The study was updated in August 2026.

It's not code review that's slowing things down. It takes time to get the thing you want, and then you have to be reasonably sure that it's right anyway, or you risk shipping a piece of crap to production and tanking your own reputation. Even on teams that rubber stamp PRs I see people pushing drafts and waiting to merge until they feel comfortable with what they've vibed out.

As a senior engineer that is struggling to give up line-by-line reviews, I agree with your take.

I've spent literal decades where one of the most valuable practices has been code reviews, for many dimensions. It is one of the best ways to ensure quality production releases. It is one of the best ways to transfer skill to junior engineers. It increases tribal knowledge throughout the engineering org by exposing others to systems they may not be working in. The number of benefits to a traditional human-centric engineering team is hard to overstate.

But I'm starting to feel that instead of starting at one extreme (heavy handed reviews) and moving towards less review, it probably makes more sense to start at the other extreme (no review at all) and move towards a brand new practice tailored for the new agentic reality.

It is looking increasingly likely that we don't need to modify code review for the agentic era, we need some completely new practice that hasn't yet been invented.

I lean towards a hybrid review process where humans accept their limitations, focus what they're good on(higher level concepts and architecture of the proposed change), and let the LLMs do the rest.

Doing a line by line review as a human is silly now.

> shipping significantly faster

If you're right (and you probably are), we're in for a wild ride further down the quality death spiral.

The era of user telemetry and conversion tracking already decoupled product design from customer experience. No need to wait for customer's to articulate their experience or desires or for discerning analysts to amalgamate and ponder their opinions. The numbers tell you everything you need to know! Just throw 3000 pounds of spaghetti at the wall every month and wait to see what sticks.

But now you really were bottlenecked on engineering.

Only 3000 pounds of spaghetti per month? And throwing more bodies at it barely even makes a difference? Oof.

Of course the solution is to just flush pureed spaghetti-slop through the hose as fast as we can. Brilliant!

What could go wrong? The users are sure to love it. We'll know because they keep clicking. That's how they tell us the truth, even as every forum fills with louder and louder complaints!

What could possibly go wrong?

> I think if there is a major change in code review culture, it's not going to happen first in established engineering teams. It'll happen in startups that adopt a no-human-review policy from jump and then maintain it as they mature.

I've been hearing this for more than a year. But if these startups are all moving at superhuman speed, a month instead of a year from idea to a shipping product, where are the products? Every time I ask that, people talk about the amazing velocity gains they experience for the thing they are about to ship. It never does, unless we count throwaway hobby projects at GitHub - by that metric, AI is adding a lot of software. And I guess never-downloaded slop-games on mobile app stores.

But if you actually go shopping for consumer or enterprise software that's camera-ready, the landscape is basically the same as in 2022.

> I think senior engineers find it very difficult to give up on line-by-line review, no matter how much better the agents have gotten.

Why do you feel like that? Agents have got better but better how? Looking at the frontier model output, I’d say the same myopic approach that causes long term issues without frequent course correction is still there.

I would reframe this as: Quality was always a significant bottleneck and is even more so with agentic coding.

Code review is where many of the design & code quality problems are observed/highlighted.

We're going to need to start putting the month in brackets after the article title, not just the year!

Both Claud and Codex would run circles around 80% of the developers I’ve worked with through my 20 year long career. This whole code review thing is a hold over that probably won’t last six months.

> I think senior engineers find it very difficult to give up on line-by-line review, no matter how much better the agents have gotten.

Our industry has spent the better part of the last two decades interviewing with brain teasers and leet code questions. Reading code is a skill that we just weren't assessing for, and lots of people aren't really good at it.

I would argue that code review is mostly theater, politics, bike shedding and pedantry.

There is a lot of work to get away from "coding" and "programing" and get down to the actual engineering that really needs to happen in the industry.

IME, the biggest source of rework cost that happens at (or after) code review is when someone notices that the code, correctly or not, implements the wrong thing. For example, the feature, as implemented, doesn’t sufficiently solve the original business problem. Or it solves it in a bad way, like because of overengineering or taking on unnecessary technical debt. I haven’t really seen much evidence of AI code review catching either of these problems.

I’ve heard people complain that human code review doesn’t reliably catch them, either. And I have seen that with my own eyes. Just not universally. Teams that have a strong culture of shared ownership and collaborating on designs regularly catch these kinds of problems in code review. If they’re really good at it, they catch them even earlier. e.g., I know planning poker isn’t fashionable anymore, but when I’ve been on teams that take it seriously they tend to be pretty good at finding minimally expensive, correct designs pretty early in the process. Which makes both coding snd code review go faster because simpler designs are easier to manage. And design flaws that do make it to code review tend to be smaller and less expensive to fix. So those teams didn't tend to bottleneck on code review in the first place.

I think that this is the kind of thing that led to folks like DORA observing that AI coding is something of a multiplier for the effects of both good and bad software development practice. If you’ve got a cowboy coding culture, throwing AI into the mix will probably just lead to agentic cowboy coding.

For me reading ai code feels a lot like that time I tried reading Joyce. I don’t understand but I’m sure it’s beautiful.

> I think senior engineers find it very difficult to give up on line-by-line review, no matter how much better the agents have gotten.

Interesting framing that casts senior engineers as the impediment. If the models are now good enough now, perhaps engineering leadership could[1] do away with code review entirely - or maximally automate code reviews. I'm sure seniors would love to get their individual productivity unleashed for performance metrics/bonus reasons.

1. I'm stating this as a hypothetical, but this "vibe-coding your core product" experiment has been repeated many times and I'm yet to hear a single success story

Imagine no code review: poof. You still aren't shipping at lightspeed because it still comes down to knowing where to shoot. Oftentimes this is like pulling teeth from various involved stakeholders.

You say small startups would be faster as they won't be doing code review. There's probably always been startups running without bothering with code review for as long as this has been a field. Bad practice? Sure, but all that matters to the business case a lot of the time is just shipping something that works alright, not the greatest piece of efficient code in the world. Moore's law has bought us a lot of slack in that.

I agree, I do like them for creating believable interaction prototypes though.

"The overall performance improvement gained by optimizing a single part of a system is limited by the fraction of time that the improved part is actually used" - Amdahl's Law

Or we could just recognize these llms crank out poorly architected bug infested slop. The coding work of these llms is just as garbage as all the other bland, flat, wtf writing they crank out with prose. Even worse because there's no logical mind behind the garbage. With bad human written code there was at least one human there thinking about it.

The hardest part always has been and always will be - a simplified vision and strategy.

Everything else is a case of project decomposition, planning and scaling up - aka this is when having code written faster is great.

The SV approach of the past decade has been about experimentation, A/B testing and data science - it has its limits.

Except the problem with this is an engine designer doesnt want to pivot to designing a stronger driveshaft, or strengthening the gears of a gearbox, or designing more efficient brakes. They're an engine builder.

Same goes for software. 20 years ago I got into software development for the love and passion of solving problems and writing code. Now I've been forced to pivot to a glorified BA/Products Manager. It sucks.

IMO it's a lot more like a new more powerful engine is now available. Sure, faster drivers/teams will run faster lap times, no doubt, and it'd be a bad idea to not use it. But, less mature teams/drivers aren't going to catch those faster teams by just slapping the new engine in and going. They are still largely held back by the myriad of other things that go into being fast on track consistently IE (driver + datalog review, and just good ol seat time), vehicle [other than engine] setup, aero tuning, pitting practice (esp fueling for endurance stuff), etc etc. All those same things that were making them slower than the fastest teams before. And even still, if you give only the slower teams the new engine, they probably still won't be competitive with the faster teams and, quite honestly, the not great drivers will probably just make the same mistakes they were making before that slowed them down, except faster.

But, even if the new engine is the golden egg, now what? You're still competing against every other team that has the golden egg... And it's right back to the driver mod, vehicle setup, etc.

That's not to say that the slower teams have any choice but to run the engine too to not be even less competitive. It's just to say that having the engine isn't gonna fix the underlying problem, no matter how much of the team you lay off to fuel the new, more power, engine.

They've updated it for the agentic code era, and have found basically the same thing. Code can be created faster but that has moved the bottleneck to review.

The study was updated twice with the last time on August 2026.

> The models involved in the main agent results were primarily Claude 3.5 through Sonnet 4.5, GPT-4.1 through GPT-5, and Gemini 2.5.

What we have today is not meaningfully better than what we had a year ago, so that isn't relevant.

By that logic, it is not ever possible to do research on LLM-assisted work.

Maybe I'm missing something, but it seems like the obvious solution is to have human accountability on the API design and tests, not on line-by-line source code. If the human writes an appropriately-scoped API and the code passes human-reviewed tests, then the implementation details don't matter so much (depending, of course, on project-specific requirements around performance and security).

Has someone presented an alternative to code Review , i mean, instead of just not doing it? Formal verification sounds expensive, asking an llm to Review another llm otput does not seem bulletproof. Is there another approach out there ? Can it exist? I've been thinking on this latelly but i can't find a solution

It was more important for giant C or fortran code bases. You will legit catch pointer errors or memory mgmt stuff, going line by line. Once Sun busted in with Java is when I really saw the shift to skim reviews. So much stuff is caught by compilers now, or SAST, that using dev time to find code smells is mostly a waste of their time. And it's not like the old way caught everything, it's just that the stuff we take for granted being caught by compilers or Sonarqube now, just used to get shipped to prod and cause abends and production support calls at 3am.

I’ve caught bugs in code reviews before, but I would agree that it’s probably not the most crucial thing you should strive to do with code reviews.

That said, I think I’ve done a tremendous amount of PREVENTING bugs from being introduced via code review. Caught all kinds of weird code patterns, duplication, wrong levels of abstraction, etc. Essentially, the kinds of places that bugs like to hide and grow in.

I still think that’s valuable, and it’s the kind of thing that models generally aren’t great at, to this day. It generally relies on having a large amount of context about the project and about what’s being solved, so if you wanted to do this via automated review, you maybe COULD, but it would probably get pretty expensive, especially on larger projects.

I review code line by line and build a mental model of the software as I review it. I find plenty of important bugs that way. PRs are an opportunity to learn (both committer and reviewer), communicate, and improve the quality of code a team ships. Sadly, many people decline that opportunity for the sake of short-term velocity.

The best code review process I've done are more like oral exams ie: the person walks through the changes and we listen as they describe it.

It's a good way to catch thinking errors, which are the insidious errors.

I think a code review should be just that. A review of the optics of code itself not the solution or anything like that. If that is important it should be done earlier in the process before coding. Even more important when coding with AI. The reviewer is not a tester. I've heard horrible stories from friends who mainly works with fixing AI generated code that junior devs pushes at a rapid pace.

Code review is like a dam holding back a huge body of water. Nobody does it perfectly and almost all software turns to shit given enough time.

Yes I have often caught stuff like "not using DB transactions in the right way" or "this is not going to be race safe" for instance from juniors just by glancing on a PR.

I find recent LLMs don't do such mistakes though, in my experience. And also LLMs will find such mistakes if asked to review, if not perfect then more predictable than humans.

No. You are on to something here. The utopia I see programmers lamenting the loss of, the one where human code review was perfect and never let any bugs slip through, never existed. I see people on this board and others complaining all the time about LLM generated spaghetti code and bug-ridden codebases. Well, I've been working those code bases for 20 years. We didn't agents to make them. LLMs are a new tool, but the problems that come with them are nothing new.

I think it highly depends on what field you're in. Personal projects that I find useful in my everyday life? Churning them out like crazy. Enterprise software? It has slowed our productivity down to a halt--part of this is due to people not knowing how to use these tools, and the other part of it is the collective AI psychosis of our leadership (I work at a Fortune 500 software company).

Organizations that use Jira have not reaped efficiency gains from AI, news at 11

Yes, I am literally inundated by cool new tools I've built that I never would have before.

I'm literally building a black-box client + server emulator for one of my favorite old MMOs, remotely, from a coffe shop in SF as I type this, on my laptop at home via Claude remote-control.

Insane.

Based on a sample size of one or…?

> the processes we love to make rarely help us

I'm going to make a potentially bold claim here. This depends _heavily_ on what the threshold of "rarely" is, and whether we actually _love_ to make these processes.

In the case of people before process, as stated in the Agile Manifesto, the process is begat _from_ the people in many cases. That doesn't mean it shouldn't be questioned, but when the chimney sweeps that built the process are saying "I don't want to go back to things that are similar to the way that gave me mesothelioma", I don't think it's fair to write them off as the minority.

You read the paper, saw that the stated collection dates ranged from January 2021 to March 2026, and are claiming the authors are lying?

Before it was all coding on its own, I asked a model for bugs in a new project, saved me the 15 minutes of new web project I waste every time trying to remember obscure first time setup things you dont touch for years in under a minute.

I suspect theres a lot of hours saved that are otherwise wasted by devs when some simple thing throws off their implementation completely. Basically we cant measure the saved developer hours by LLMs because it could be literally anything.

This stuff is still 90% garbage hype. See also:

https://www.youtube.com/watch?v=NnV_cWeoo5Q&list=LL&index=3

Ars Technica in general has also been catering to their knee-jerk anti-AI audience. I used to find their comments section somewhat interesting but now it's the same old tired, outdated dunks on AI.

You are free to not comment in the future!

I would be more concerned with playing video games half the day than whether the PMs are slowing you down by figuring out the requirements

Listen man. Let me give you a tip.

Don't take it seriously.

Efficiency is not your job, until you can 1) solve it, 2) present the solved and implemented problem to senior leadership in one fell swoop.

Corporate America is a giant slush fund and this is how it keeps going.

Be grateful!

What kind of professional would spend 4 hours of their time per day playing video games?

The true professionals spend those 4 hours making snarky Hacker News comments.

A rocketship pointed in the wrong direction is not going to get to mars. Steering is probably the next evolution of code review