Martin Fowler said it plainly at the second Future of Software Development Retreat last week: the bottleneck has shifted. It’s not code generation anymore. It’s verification.

And buried in that observation is a discipline that most engineering teams are completely ignoring: harness engineering.

Not prompt engineering. Not model fine-tuning. Not even agent orchestration in the abstract sense. Harness engineering — the deliberate design of the loops, constraints, scaffolding, and verification systems that wrap AI output before it reaches production.

I want to be direct: if your team doesn’t have someone thinking about harness design, you’re accumulating invisible risk at a rate your incident dashboard can’t yet see.

What Is a Harness?

In test engineering, a test harness is the infrastructure that runs tests: the fixtures, mocks, runners, reporters. It’s not the tests themselves — it’s the system that makes testing reliable, repeatable, and trustworthy.

An AI harness is the same concept applied to AI-in-the-loop workflows. It’s not the AI model. It’s everything around the model that ensures the model’s output is:

• Verified before it reaches downstream systems
• Constrained to the boundaries of what’s acceptable
• Recoverable when the model produces wrong output
• Observable so you can track quality over time

Here’s the simplest mental model: if you remove the AI from your workflow and insert a human expert instead, what infrastructure would you build around that human to ensure quality? That infrastructure is your harness.

Why Harness Engineering Matters Now

Two years ago, AI in software teams meant autocomplete. The “harness” problem was trivial — if the suggestion was wrong, the developer just ignored it.

Today, AI is generating entire features, writing migration scripts, proposing architectural changes, and in some cases committing and deploying code autonomously. The harness problem is now a production risk problem.

The Anthropic Data Point

Anthropic just published something remarkable: they removed 80%+ of Claude Code’s system prompt for Claude 5 generation models with no measurable regression in quality. The implication is not “the model got smarter so you need fewer instructions.” The implication is that how you structure context matters more than how much context you provide.

That’s harness thinking. They’re designing the system around the model, not just the model itself.

The Incident Pattern

In the teams I advise, AI-related production incidents cluster into three root cause categories:

1. Unverified AI output reaching production (45%): The AI generated something plausible-looking but wrong, and no verification step caught it.
2. Context not available to the AI (35%): The AI didn’t have access to the information it needed — architecture docs, business rules, system constraints.
3. Recovery path not designed (20%): The AI produced bad output, the system had no graceful degradation path, and the failure cascaded.

All three are harness failures. None of them are model failures.

The Five Components of a Production AI Harness

1. Input Constraints Layer

What goes into the AI matters as much as what comes out. A good harness defines:

• Context documents: CLAUDE.md, ARCHITECTURE.md, domain-specific constraints that the AI must receive
• Prohibited inputs: What not to pass to the model (PII, secrets, data that shouldn’t leave your perimeter)
• Scope boundaries: What problem space the AI is allowed to work in

# CLAUDE.md — Input Constraint Example

## Scope
You are helping with the payment service only. Do NOT:
- Modify authentication code
- Access user PII fields directly
- Suggest architectural changes without flagging for human review

## Context Required
Before solving any problem, read: ARCHITECTURE.md, PAYMENT_CONSTRAINTS.md

2. Output Verification Layer

Every AI output that matters should pass through verification before acting on it. Verification can be:

• Syntactic: Does this parse? Does it compile?
• Semantic: Does this implement the stated intent?
• Constraint: Does this violate any rules in ARCHITECTURE.md?
• Behavioral: Does this work correctly on a representative test suite?

The key insight from Anthropic’s harness engineering work: verification isn’t just testing. It’s a loop where the output feeds back into the system with new context if it fails.

async def verified_ai_generation(prompt: str, constraints: dict) -> str:
    max_attempts = 3
    for attempt in range(max_attempts):
        output = await ai_generate(prompt)
        
        violations = check_constraints(output, constraints)
        if not violations:
            return output
            
        # Feed violations back as context for next attempt
        prompt = f"{prompt}\n\nPrevious output violated: {violations}\nPlease fix."
    
    raise HarnessVerificationError("Could not generate compliant output")

3. Progress Artifact Pattern

For long-running AI agents that span multiple context windows (a growing problem as agentic workflows become standard), you need a progress artifact pattern.

Instead of trusting the AI to maintain context across sessions, design the harness to maintain state externally:

agent-progress.md  ← Maintained by harness, not AI
  ## Completed Steps
  - [x] Analyzed requirements (2026-07-27T10:23:00Z)
  - [x] Generated schema migration (sha: abc123)
  
  ## Current Step
  - [ ] Implementing service layer (started: 10:45Z)
  
  ## Constraints Applied
  - No direct DB access from service layer
  - All mutations must go through repository pattern

The AI reads this at the start of each session. The harness writes to it after each verified step. If the AI goes off track, the harness detects it via the last completed step and can restart from a known-good state.

4. Observability Integration

You cannot improve what you cannot measure. AI harnesses need observability that humans never needed:

• Attempt count per task: How many tries before the AI produced acceptable output?
• Context utilization: Which context documents is the AI actually using?
• Constraint violation rate: How often does AI output violate defined constraints?
• Fallback trigger rate: How often does the system fall back to human review?

In our stack, we push harness metrics to the same observability platform as application metrics. An AI that suddenly starts requiring 3+ attempts for tasks that previously took 1 is a signal — of a context problem, a constraint change, or a model behavior shift.

5. Graceful Degradation Path

Every AI-in-the-loop workflow needs a designed failure path. If the AI cannot produce verified output within the attempt budget:

• Does the workflow pause for human review?
• Does it fail open (proceed without AI enhancement) or fail closed (halt until resolved)?
• Who gets notified? What’s the SLA?

This is standard reliability engineering applied to AI. The difference is that AI failures are often silent — the system produces output, it’s just wrong. Your degradation paths need to account for silent failures, not just timeouts and exceptions.

Practical Harness Design: A Case Study

Here’s a real harness I designed for automated API documentation generation:

Input Layer:
  - API source code (scoped to public endpoints only)
  - Existing doc standards (DOCUMENTATION_STYLE.md)
  - Known-bad example patterns to avoid

Generation:
  - Claude generates documentation draft
  - Attempt budget: 3 tries

Verification Layer:
  - Check 1: Does every public method have a doc block? (Syntactic)
  - Check 2: Do code examples in docs actually compile? (Behavioral)
  - Check 3: Are all parameter descriptions present? (Semantic)
  - Check 4: No internal implementation details exposed? (Constraint)

Output:
  - Pass: Submit PR with generated docs
  - Fail after 3: Create issue with constraint violations, assign to human

Observability:
  - Attempt count → Grafana
  - Verification failures by type → Grafana
  - Human fallback rate → Weekly review metric

After three months, attempt count dropped from 2.1 average to 1.3 average as we refined the input constraints. Human fallback rate dropped from 18% to 4%. The docs quality improved not because we changed the model — we didn’t. We improved the harness.

Building the Harness Engineering Discipline

Harness engineering is not AI engineering and it’s not test engineering. It’s a hybrid discipline that requires:

1. Deep system knowledge: You can’t constrain what you don’t understand
2. Failure mode thinking: What can go wrong, and what’s the impact?
3. Feedback loop design: How does the system improve over time?
4. Observability instinct: What do you need to measure to know if it’s working?

On a practical level, in a team of 8 engineers, you probably need one person whose primary focus is harness design for your most critical AI workflows. Not full-time — but intentionally, not accidentally.

The Trajectory

Fowler’s observation from the retreat is worth sitting with: we face an apprenticeship crisis as AI replaces the junior-engineer entry path. But the harness engineering discipline is one place where experienced engineers have irreplaceable value.

You need system knowledge to write good constraints. You need failure mode experience to design good recovery paths. You need production wisdom to know what to observe.

This isn’t a role AI will automate soon. It’s the role AI makes necessary.

---

If you’re building harness engineering practices at your company, I’d love to compare notes. Reach out on LinkedIn.