I wrote about why tech leads should stop writing prompts and start writing loops a couple weeks ago — the decision framework for when to automate a manual prompting habit. What that post didn’t cover is the part that actually breaks in production: which loop architecture you pick, and what happens when you pick wrong. A recent deep-dive from Data Science Dojo traces that evolution in detail, and it’s worth a tech lead’s time because most teams are still running Generation 1 mistakes with Generation 4 tooling.
The Loop, Reduced to Two Things
Strip away the tooling and an agentic loop needs exactly two components: a trigger — event-based, scheduled, or human-initiated — and a verifiable goal. The agent then cycles through five stages until the goal is met:
- Perceive — ingest input: the user’s goal, a tool’s result, an API response
- Reason — figure out what the input means and what options exist
- Plan — select the next action
- Act — execute it: call a tool, write code, query a database
- Observe — receive the result and update its understanding
That cycle is the whole engine. Everything that follows in this post is refinements bolted onto one of those five stages to fix a specific way it broke in practice.
Four Generations, Each Fixing the Last One’s Failure
| Generation | Era | What it added | What it fixed |
|---|---|---|---|
| 1 — Proof of concept | Mar 2023 | AutoGPT: break a goal into sub-tasks, give it web + file tools | Proved demand — and immediately hit infinite loops and runaway API bills |
| 2 — Academic frameworks | 2022–2023 | ReAct (reasoning traces beside actions, +34% on ALFWorld), Reflexion (self-critique before retry), Plan-and-Execute (separates planning from execution, 3.6x speedup per LangChain) | Gave the loop a way to reason about its own actions instead of just chaining them |
| 3 — Architectural patterns | 2024 | OODA loop (adds an explicit “Orient” step), Microsoft Magentic-One’s inner/outer dual loop, multi-agent supervisor orchestration | Stopped agents from getting stuck re-trying the same failed strategy — at the cost of ~15x more tokens for multi-agent setups |
| 4 — Practitioner tooling | 2025–2026 | Ralph Loop (resets context every iteration, uses the file system as memory), native /goal in Claude Code and Codex with a separate evaluator model, Boris Cherny’s parallel-workflow pattern (5+ simultaneous Claude sessions sharing state via CLAUDE.md) | Solved context overflow and premature exit — the two failure modes that made long-running loops unreliable |
The pattern across all four generations: someone hits a specific, nameable failure, and the next generation is a targeted fix for exactly that failure — not a general improvement. That’s useful to know, because it means you should pick your architecture based on which failure mode you’re actually exposed to, not by defaulting to whatever’s newest.
Memory Is the Part Everyone Gets Wrong First
Production loops need memory, and there are four distinct kinds in play — conflating them is where most first attempts go wrong:
- Episodic — a record of what the agent already tried and what happened
- Semantic — structured domain knowledge: your architecture decisions, your API docs, the stuff a senior engineer just “knows”
- Vector — similarity-based retrieval for pulling in contextually relevant snippets
- File-based — state that lives on disk and survives a context reset (this is the Ralph Loop’s whole trick)
Boris Cherny’s CLAUDE.md pattern is semantic memory done right: a file that persists across every session and every context reset, so the agent doesn’t relearn your codebase’s conventions every single run. If your team’s loops keep making the same class of mistake across sessions, you’re missing semantic memory, not a smarter model.
Guardrails Aren’t Optional — They’re the Product
This is the part that separates a loop engineering practice from a demo. Every production loop needs, at minimum:
- Hard iteration caps — a loop with no ceiling is a bug waiting for a bad day
- Token and cost budgets — enforced at the loop level, not discovered on the invoice
- No-progress detection — if state hasn’t meaningfully changed in N iterations, stop and escalate
- Circuit breakers on tool calls — one documented production incident had a single tool called 400 times in five minutes
- Verifiable termination criteria — “looks done” is not a termination criterion; a passing test suite is
- Human checkpoints before irreversible actions — anything touching production data, payments, or deploys gets a gate, full stop
Skip these and you get the failure modes that show up in every postmortem of a loop gone wrong: infinite loops from unverifiable goals, goal drift from ambiguous specs, context overflow degrading reasoning quality over a long session, and silent failures — confident-sounding output with zero actual progress.
The Token Bill Is a Design Decision, Not an Accident
| Setup | Token multiplier vs. a standard chat session |
|---|---|
| Single agent, ReAct-style loop | ~4x |
| Multi-agent supervisor orchestration | ~15x |
One documented case reached $1.3 million in monthly token spend. That’s not a cautionary tale about AI being expensive — it’s a cautionary tale about deploying multi-agent orchestration for a problem a single ReAct loop would have solved at a quarter of the cost. Token multiplier should be a line item in your architecture decision, not something finance discovers in Q3.
Which Pattern to Actually Use
The selection guidance holds up against what teams are reporting in production:
- Start with ReAct. It’s the default for a reason — most tasks don’t need more architecture than “reason, then act, then observe.”
- Add Reflexion when the failure mode is quality, not scope — the agent completes the task but gets details wrong on the first pass. Self-critique before retry fixes that class of error specifically.
- Reach for Ralph Loop when the task is context-heavy — long-running migrations, large refactors, anything where the context window itself is the bottleneck rather than the reasoning.
- Only go multi-agent when you’ve measured a concrete improvement that justifies 15x the cost. “It seems like it should help” is not a measurement.
What Tech Leads Should Do This Week
Audit your current loops — or your team’s manual re-prompting habits that are loop candidates — against this checklist, not against whichever architecture is trending on X this week:
- Does it have a verifiable termination criterion, or does it stop when it “looks done”?
- Does it have a hard iteration cap and a cost budget enforced in code, not in a Slack reminder?
- Is your memory strategy matched to the failure you’re actually seeing — semantic memory for repeated conventions, file-based for context overflow, vector for retrieval?
- Would ReAct alone solve this, or did you reach for multi-agent because it’s the interesting architecture rather than the necessary one?
Loop engineering isn’t a single skill you either have or don’t. It’s a stack of specific fixes for specific failures, and the job of a tech lead is knowing which failure you’re actually fixing before you add the next layer of architecture on top.