Series · Shipping with AI
- 01Vibe coding: the good, the bad, and the ugly
- ▸ 02The unruled space
Journal
The unruled space
Agents write fast, another agent approves, a human "guides" - and last Tuesday that workflow shipped wrong math and a copy-pasted fix before lunch. We tested the tools, the rules files, and the reviewers. Here's what actually holds, and what's guarded by nothing at all.
Last week we wrote about vibe coding in the abstract. This week we want to show you a single afternoon of it, from our own workshop - because the abstract version was too kind, including to us.
Like most teams building software in 2026, we run coding agents. Several at once, across projects: dispatch a task, check the result, redirect, move on. The economics are real and we won't pretend otherwise. But the popular version of this workflow - agents write, another agent reviews, a human "guides" - has a hole in it, and last Tuesday the hole introduced itself twice before lunch.
Two catches in one afternoon
The first: an agent was writing scroll behavior for a client interface and quietly invented its own offset arithmetic. The math was wrong. Not obviously wrong - plausibly wrong, the worst kind. Its own tests would have passed, because the tests asserted the same wrong math. An AI reviewer would have approved it, because plausible-looking calculations are precisely what these models are worst at doubting. The only detector that fired was an engineer reading the code as it was written.
The second: a form with two comboboxes. The first combobox had text overflowing its boundary. We asked the agent to fix the second one. It fixed the first. We corrected it; it fixed the second - by copy-pasting the same patch. Then the field showed a raw database id instead of a label; that got fixed too, again pasted per-instance. Three fixes, all local patches, while the correct change was one edit in a shared component. The agent had its own first fix sitting in context and still duplicated it. A working patch in view didn't nudge it toward "abstract this" - it handed it a template to paste.
That's the pattern worth writing down: agents do not generalize on their own initiative, even with everything in front of them. "Fix the second one" gets you the second one fixed, by the cheapest available route. The thought same fix twice means the bug lives upstream does not emerge. It has to be demanded, or it has to be caught.
Can tooling catch it? We tested.
Our first instinct was machinery. Duplicate detectors like jscpd are free, deterministic, and fast, and they do catch copy-paste clones - even reformatted, re-commented ones. So we ran an experiment: the same retry helper written twice, once copy-pasted, once rewritten from scratch the way an agent actually rewrites things - different loop shape, different names, different error handling.
The copy-paste was caught in four milliseconds. The rewrite was invisible. And when we tuned the matching aggressively enough to catch rewrites, retry logic started "matching" a currency formatter. Miss the rewrites or drown in false positives; you get to pick one. Full mechanical coverage of the agent wrote this again, slightly differently does not currently exist, whatever the tooling landscape claims.
Then the rules leaked too
Second instinct: rules. We keep instruction files for our agents - ours is about a hundred lines, disciplined, nothing bloated. One rule states plainly: never write raw form fields inline; use the form components. Those two comboboxes from Tuesday? Written inline. The rule was right there. Ignored.
Here's the uncomfortable property of prose rules: they're probabilistic. An instruction in a context file buys you something like 85–90% compliance per opportunity, because the model's attention is on the task in front of it and background constraints lose that fight. At agent volume, 90% doesn't mean "mostly fine." It means violations arrive on schedule.
What actually holds
After enough afternoons like that one, a shape emerges. It's less exciting than the tooling brochures and it works.
Anything that matters and can be checked becomes a check, not a sentence. Our form-component rule should never have been prose; it's now a lint rule that hard-fails on raw form elements outside the components directory. A prose rule is a suggestion the model may weigh. A failing check is a wall - and the error message feeds back into the agent's own loop, so it corrects itself without anyone watching. The same goes for the rule that every bug fix ships with a regression test that fails on the old code. That one is non-negotiable; it's the only thing that stops the same bug returning three releases later.
The codebase teaches louder than any instruction file. Agents imitate surrounding code far more reliably than they obey rules. If a repository has one clean combobox component used in ten places, agents copy the pattern; if the repository itself contains inline fields, the rule loses to the evidence. Every consolidation we do is a rule written in the only language the model reliably follows.
Tight briefs shrink the space where the model freelances. "Add filtering" leaves a thousand decisions to a contractor optimizing for looks done. "Add a filter using FormCombobox, same pattern as the products page" leaves almost none. Two sentences of specification remove most of the risk before any code exists.
And the rest gets read - by people, deliberately, where nothing cheaper works. Not every line; at agent volume that's arithmetic, not virtue. But hand-rolled calculations, auth, money, data migrations, anything with a formula: human eyes, every time, because Tuesday demonstrated that for this class of defect, human eyes are the only working detector.
The part the hype skips
Everything without a check, a pattern, or a reviewer is guarded by nothing except the model's default judgment - and its default judgment is a competent-sounding contractor optimizing for the smallest edit that satisfies the last message. You cannot enumerate rules for an infinite space. No stack of tools closes it. What you can do is make every catch permanent: the wrong scroll math became a regression test, the inline combobox became a lint wall, the pasted fix became a consolidation and a diff check. The defect rate falls and keeps falling. It never reaches zero. It never reached zero when humans wrote everything by hand, either.
So the honest summary of AI-assisted development, mid-2026: the speed is real, and so is the supervision. Engineering attention became the most expensive resource in the system, and the whole craft now is spending it where nothing cheaper works. Teams that skip that part ship fast for a few months - and then they're searching for someone to inherit the codebase. Lately, that's the work walking through our door.