Two essays from HumanLayer’s Dex Horthy, read together, make a stronger argument than either does alone. “Advanced Context Engineering for Coding Agents” (ACE-FCA) is the practitioner’s how-to: the exact workflow that got a small team shipping 35k lines of production Rust in a single day. “Why Software Factories Fail” (WSFF) is the theory underneath it: a plain explanation of why today’s models can’t be trusted to run unsupervised, no matter how good the harness is. Read in order, they form a single argument: the industry’s “just write more loops, let the agent run lights-off” narrative is optimizing for the wrong thing, and the actual fix is a specific, learnable discipline — not a better prompt, not a bigger context window, not the next model.

The Theory: Why “Lights-Off” Doesn’t Work

WSFF’s core claim is uncomfortable for anyone who wants AI coding to be a solved problem: this is not a skill issue, it’s a model-training issue.

Coding models get trained via RL against benchmarks like SWE-bench: an agent gets a bug report, writes a patch, and the patch is scored 1 or 0 based on whether tests pass without breaking other tests. That reward signal has a hole in it big enough to drive a truck through: there is no penalty for eroding codebase maintainability. A model that wraps everything in try/catch, casts its way past the type system, or duplicates logic across eleven files scores exactly the same as a model that writes the clean version — as long as the tests go green. Millions of RL iterations later, you’ve optimized hard for “makes tests pass” and not at all for “doesn’t make the codebase harder to work in six months.”

That gap explains a pattern a lot of teams have lived through firsthand: models got dramatically better at solving one-off problems and greenfield builds over the last year, but “not much better” at maintaining codebase quality over time — because there’s no fast, reliable way to score that during training. Test-passing is a same-day signal; bad architecture is a cost that shows up weeks or months later, and there’s no way to backpropagate that incident to the design decision that caused it.

Horthy’s team tried the fully lights-off factory for real, in mid-2025 — background agents handling everything, no human reading code. It worked until it didn’t: eventually a bug surfaced that no amount of prompting could untangle, because nobody had been reading the code that led up to it. By the third recurrence, it was cheaper to rewrite the system from scratch than keep debugging what the agents had built.

The Fix: Turn the Lights Back On

The fix isn’t “review more code” in the old sense — that doesn’t scale to agent-speed output either. It’s front-loading structure before code gets written, the same instinct that made human software teams do planning and architecture reviews decades before AI existed, applied at the four checkpoints where a wrong turn is cheapest to catch:

  1. Product Review — a short doc pinning down the actual user problem and what success looks like, with rough HTML mockups instead of prose descriptions where the UI is in question. Not everything needs this — a copy tweak or an obvious bug fix skips straight to the agent.
  2. System Architecture — how services, endpoints, schemas, and stores talk to each other, expressed as sequence diagrams and contract shapes, without descending into implementation detail yet.
  3. Program Design — the step most teams skip, and the one Horthy calls “criminally underemphasized”: call-stack trees, file-tree diffs, and method signatures for the shape of the code, decided before anyone writes the implementation. Every one of these is a decision you’d otherwise be making implicitly during code review, at the most expensive possible point to change your mind.
  4. Vertical Slices — instead of building horizontally by layer (migrations → services → API → frontend, which produces nothing testable until the very end), build one thin path through the whole stack at a time, testable in a browser or with curl at every step.

Not every task earns all four. Horthy’s rough distribution: ~40% of tasks get one-shot with light feedback, medium tasks collapse product/system design into one document, and only large or genuinely risky work gets the full four-phase treatment.

The Companion Practice: Context Is the Only Lever You Have

ACE-FCA supplies the mechanism that makes the four-phase process actually work turn to turn: frequent intentional compaction. The premise is blunt — an LLM call is a stateless function, and the context window is the only thing that determines output quality. Optimize it for correctness first, completeness second, size third. The worst things that can happen to a context window, in order, are wrong information, missing information, and noise.

The concrete workflow is three steps, deliberately kept separate so no single context window has to hold everything at once:

  • Research — understand the codebase and how information flows, with no attempt yet at a fix.
  • Plan — the exact steps, files, and edits, precise about testing and verification per phase.
  • Implement — step through the plan phase by phase, re-compacting status back into the plan doc as you go.

The leverage argument underneath this is the one worth remembering even if you forget everything else: a bad line of code is a bad line of code. A bad line of a plan can produce hundreds of bad lines of code. A bad line of research — a wrong assumption about how the codebase works — can produce thousands. So human review effort belongs at the research and plan stage, not spread evenly across the pipeline. Reviewing a 200-line plan is a fundamentally different, cheaper act than reviewing a 2,000-line diff after the fact, and it catches the same class of mistake earlier.

What This Means for a Tech Lead

Put together, the two essays converge on one operating principle: the leverage point in agentic coding moved from “write the code” to “write the spec, and decide who reviews it and when.” The job isn’t disappearing, it’s moving earlier and getting more concentrated — which is exactly why a tech lead’s judgment matters more here, not less.

Implementation Guide

At the company

  1. Kill any “lights-off” pilot quietly running today. If any part of your pipeline routes agent output to production without a human reading it, that’s the exact failure mode both essays describe. It will work for months, then fail expensively on the one bug nobody was tracking.
  2. Match review depth to task size, not to “it’s AI so review everything the same way.” Adopt Horthy’s rough split: trivial tasks (copy, obvious repro) go straight to the agent; medium tasks get one combined product/architecture doc; only large or high-risk changes get the full four-phase treatment. Forcing every task through the heavy process burns the goodwill you need for the cases that actually warrant it.
  3. Add “Program Design” as an explicit step if your team currently jumps from architecture straight to code. This is the gap most teams have — not because it’s hard, but because it feels like unnecessary ceremony until you’re the one debugging the elventh copy of duplicated logic. A call-stack diff and a few type signatures cost minutes; they’re the cheapest place left to catch a wrong turn.
  4. Rebuild your PR review habits around research and plan docs, not just the diff. If your team already does async doc review (Notion, GitHub discussions, whatever), extend it one step earlier — review the plan the way you’d review the PR, since a plan-level mistake is the more expensive one to catch late.
  5. Instrument for the failure mode, not just velocity. Track rework rate on merged PRs and incidents-per-agent-PR, the same signals Faros AI’s report flagged industry-wide (more review comments, more incidents, more PRs merged with no review). If those numbers are climbing while your team celebrates “AI shipped X% of our code,” you’re in the failure mode WSFF describes, not the success case.
  6. Budget review time as engineering time, not overhead. Both essays are explicit that this discipline costs real hours up front — 30 minutes of planning to save hours of review is the trade, not a free lunch. If leadership expects agentic coding to be free of that cost, that expectation needs correcting before the pipeline gets built, not after the first outage.

For your own projects

  1. Adopt research → plan → implement even solo. The leverage argument holds at team size one: a wrong assumption in your own head, uncaught before you start implementing, costs you the same multiplying effect it costs a team.
  2. Write the plan as a file you re-read, not a chat you scroll back through. Compacting your own progress into a plan.md after each phase is the individual-scale version of “frequent intentional compaction” — it’s what lets you pick a project back up after a break without re-deriving where you left off.
  3. Treat vertical slices as the default, not horizontal layers. Building database → service → API → frontend in stack order means nothing is testable until the very end. Building one thin end-to-end path first — even with a mocked layer — means you catch a wrong direction after twenty minutes, not after two thousand lines.

The Guardrail

Neither essay argues for going back to writing everything by hand — both authors are shipping enormous amounts of agent-written code, faster than before. The guardrail is narrower and more specific: the parts of software development that require judgment about tradeoffs — what to build, how the pieces fit, what the code should look like before it’s written — still need a human in the loop, because no training signal today rewards models for getting those parts right. Skip that judgment and you’re not moving 10x faster; you’re borrowing velocity against a codebase you’ll eventually have to pay back, with interest, in an outage or a rewrite.

This week: pick one feature or fix currently headed straight from prompt to merge with no plan document, and insert a research + plan step before implementation — even a rough one. Compare how the review felt against your last unreviewed agent PR.

Source: Why Software Factories Fail and Advanced Context Engineering for Coding Agents, both by Dex Horthy / HumanLayer.

Export for reading

Comments