Was This Different Two Years Ago?

There is a list of complaints I hear more and more often, and it has become oddly consistent. There is slop everywhere. Reviews are either rubber stamps or drowning. The architecture is drifting, and everyone is moving too fast to notice. Every so often it goes further than engineering, and the leadership of the organisation ends up in the same sentence as the tooling.

The common thread is that the LLM gets named as the cause. I don’t think it’s the cause of any of these, at least not directly. What it does is magnify every one of them, because the rate of output has changed.

The question

When someone tells me about one of these problems, I ask them the same thing every time: “was this different two years ago?”

I’ve asked it enough times now to see a pattern. My sample is the people I talk to about this (mostly engineers and engineering leaders, which adds up to a lot of conversations but is not a survey). Once they think about it, the answer is nearly always some version of no. The problem was still there - it’s the speed at which it happens that changed.

That isn’t a small distinction. If the problem existed two years ago, the model didn’t create it, and taking the model away won’t remove it. You would just go back to hitting it at the old rate, which is the rate everyone had quietly learned to live with.

The arithmetic

My own numbers make this concrete. A year or two ago, a normal week for me was three or four pull requests for larger changes. Now it’s thirty or forty, and on a long and intense week it has gone past a hundred. That is an order of magnitude, not a productivity bump.

Now assume my rate of bad work stays exactly where it was. If one of those pull requests a month was not great back then (and there was always one), at the new volume that’s ten or twenty a month. The people around me don’t see a percentage. They see ten or twenty things that need fixing, reviewing again or unwinding every month, where there used to be one.

I think that goes for everything, and everyone. We have always had people where 90% of what they produce is a pile of shit. We’ve always had the other extreme too, the people where it’s more like 10%. But nobody has ever been at zero, me included; there has always been shit generated. (I made the longer version of that argument in Nobody Reviews the Assembly: the baseline was never zero.) What has changed is the sheer amount of it, because the amount of everything went up.

Slop

What people are complaining about is rarely code quality as such. The word they reach for is slop, and it’s the right one, but slop has always existed. What changed is what it costs to produce.

Slop used to take real effort. Somebody had to sit down and write the thing by hand, over hours or days, which kept the amount of it down to something an organisation could absorb. That effort was not wasted either, because producing it was how an inexperienced developer became an experienced one. You built something bad, somebody senior read it and told you exactly why it was bad, and you didn’t build that particular bad thing again.

So slop was bearable, and it was a good deal better than bearable. Reviewing it was one of the best mentoring mechanisms an organisation had. I’ve argued before that code review was never much of a defect filter, and this is where it actually paid: not in catching the bug, but in being the place somebody explained the reasoning to you at the moment you most needed it.

Now the effort is gone, and both halves of that go with it. The volume arrives without anyone having struggled to produce it, so the learning that used to be attached to it doesn’t happen, and the review that used to be a teaching moment is just a queue.

That is the one place where I think the rate itself has genuinely changed, rather than only the count. The model hands out capability without the understanding that capability used to require, so someone working well past their depth can now produce something plausible that they can’t defend, and I don’t have an answer to it yet (ZFN-28Open problem · currentZFN-28 — Capability without understanding: brute-force LLM PRsAn open problem: people brute-force PRs with LLMs in domains they don't understand, taking on more than their knowledge supports — and the struggle that used to teach them is smoothed away. How do we stop un-understood code without killing learning or banning a good tool?Open ZFN-28 → is filed as an open problem for exactly that reason). But even there, whether it gets caught comes down to the same things it always did: whether someone clearly owns that part of the system, and whether the change had to cross a boundary where somebody would notice.

The same wall

None of this is new to me personally. Every time I’ve found a way to substantially increase my own output inside an organisation, I’ve hit the same wall: the organisation struggles to keep up. The change is ready long before the organisation is ready to take it, and the bigger the organisation gets (and the more things it has going on) the worse that gets.

What’s different now is that it isn’t just me, or the handful of people who had found some way to go faster. Everyone’s rate went up at roughly the same time, organisations have more going on than ever, and the rate of change has gone up everywhere at once. That is what’s dragging poor management and poor engineering organisation into the light. Those problems were survivable at the old pace, and they aren’t at this one.

Cross-cutting changes

Where it hurts most is any change that has to cross the product. A real feature often touches several parts of it at once:

• the frontend
• the backend
• billing
• the operations and management side of it

If there aren’t good, well-defined interfaces between the parts of the organisation that own those, and it isn’t clear who owns which code, that change is miserable to make. Not because it’s hard to write - the model will happily write all four parts in an afternoon - but because nobody is sure who has to agree to it, whose review actually counts, or whose problem it is when it breaks at two in the morning. That part was always slow (it’s just that the writing used to be slow too), so now the slow part is all that’s left, and everyone is staring at it.

I wrote down what I’d do about it back in June: govern the contract between teams, not the code inside themField Note · currentZFN-47 — Govern the contract between teams, not the code inside themTeams own services end to end; one team owns the gateway that dispatches to them. Govern exactly one thing centrally — the contract at the boundary (schema, identity, errors, idempotency) — and enforce it at runtime. Don't mandate libraries; ship them as an opt-in blueprint.Open ZFN-47 →. Draw the organisation so each team owns its part end to end, and put the governance on the boundary between teams (the schema, the identity model, the error semantics) rather than in everyone’s pull requests. That note was about running teams of people. It applies at least as much to an organisation trying to absorb what a few people can now produce with these tools.

The wrong lever

My advice to anyone in this situation is to stop thinking of the LLM as the problem.

I understand the instinct. The model is the thing that changed, so it’s the thing people reach for: rules about how you’re allowed to use it, limits on what it’s allowed to touch, more process wrapped around it (often all three at once). But the model is only what’s giving people the ability to move faster. Throttle it and you get back to the old rate, with all the old problems still in place (just arriving less often), and you’ve thrown away the speed to get there.

What needs to change is how we interact with the organisation: its structure, who owns what, and how work gets across the boundaries between teams. That is engineering leadership’s job, and it is more vital now than it has ever been, because those things now set the pace for everybody.

Engineering leadership

In fast development organisations, great engineering leadership has only really become important once the teams reached a good level of performance. Before that point there is always something else slowing them down. Most organisations spend forever spinning, blaming things (and the LLM is only the latest thing to blame), and never pull back far enough to start implementing the long-term engineering leadership strategies that actually work. You only feel the absence of those once the teams are good enough that the organisation is the thing holding them back.

Now the speed has improved for everyone, and that point has arrived everywhere at once. Teams that would have taken years to get fast enough to hit the organisation are hitting it today, so what used to be a concern for a handful of high-performing organisations has become relevant to all of them.

That’s not to say nothing in engineering leadership has changed. The weights on a lot of the variables in organisational structure and process have been heavily modified: writing the code got cheap, while agreeing on what to build and getting it across the organisation didn’t. No doubt that will introduce new changes further up the leadership stack too. But what was already true about good engineering leadership is still true. It just applies to everyone now, rather than only to the organisations that had got fast enough to need it.

Small teams

The other side of this is the part I’m genuinely excited about! You can do so much more with a very, very small team now than you could even a year or two ago (the arithmetic above cuts both ways). But that team has to be empowered to actually do it, and it has to sit inside an organisation that isn’t causing it massive slowdown, so it can move at the rate the tools now allow.

That isn’t what I’m seeing. What I’m seeing is organisations struggling to enable their developers to move at the speed they’re capable of, and honestly I’ve always seen that. It just didn’t matter much before, because most people didn’t move that fast, and mediocre performance was seen as good output. An organisation that could only absorb mediocre output never had to find out, because mediocre output was all it was being asked to absorb.

Now most people can move that fast, and the throttle is being put on the developer. Which means the bottleneck isn’t really the developer any more. It’s the organisation.

My guess is that this ultimately leads to smaller, very focused products powered by a very small number of people. You don’t need a large organisation to perform and execute the way you used to. That’s a prediction rather than something I can show you yet, and I’m a lot more confident about the direction than about how quickly it happens.

Nothing new

None of these are new problems. Unclear ownership, blurry interfaces between teams, reviews nobody has time for, architecture that drifts because nobody is accountable for it: every one of them has existed for as long as I’ve been doing this, and I started in the late nineties.

What’s changed is that they’re far more obvious now, and far more painful. The LLM didn’t create them. It took away the slow pace that let us live with them.

← All writing