The numbers came in last week and they’re not pretty: pull requests are up 20% year-over-year at companies with AI coding tools, but incidents per PR rose 23.5% in the same period. More code, more bugs, fewer eyes per line.

I’ve been a Tech Lead for 15+ years, and this is the first time I’ve felt the code review process actively working against us. Not because engineers are lazy — they’re not. But because the assumptions baked into how we review code were written for a world where humans wrote every line.

That world is gone.

The Real Problem Isn’t Volume, It’s Signal-to-Noise

When a human writes 200 lines of code, you know roughly what they were thinking. The code reflects their mental model, their habits, their understanding of the codebase. You’re reviewing a person’s reasoning.

When an AI writes 200 lines, you’re reviewing a probability distribution’s output. The code can be stylistically flawless and architecturally wrong at the same time. It passes linters, satisfies tests, and introduces a subtle race condition that only triggers under specific load patterns.

The AI doesn’t know what it doesn’t know. And neither do you, if you’re reviewing at the same pace you reviewed human code.

What I’ve Observed Across My Team

After three months of heavy AI coding tool adoption (we’re using Claude Code and Cursor in parallel across two squads), here’s what I’m actually seeing:

Before AI tools (Q1 2026):
  Average PR size: 180 lines
  Review time per PR: 45 min
  Incidents from merged PRs: 2.1/month

After AI tools (Q2 2026):
  Average PR size: 340 lines  ← +89%
  Review time per PR: 38 min  ← -16%
  Incidents from merged PRs: 3.8/month  ← +81%

The engineers were reviewing faster because the code looked cleaner. But they were missing things. The AI generates code that reads well but may embed assumptions that don’t match the system context.

The Five Categories of AI-Generated Code Failures

After reviewing 400+ PRs from AI-assisted sessions, I’ve noticed failure patterns cluster into five categories:

1. Context Blindness

AI generates code that’s correct in isolation but ignores existing system conventions. Example: adding Redis caching in a service where we explicitly don’t cache because of compliance requirements. The AI didn’t read ARCHITECTURE.md. It generated valid caching code that violated a business rule.

// AI generated — looks fine, fails compliance
public async Task<UserData> GetUserAsync(string userId) {
    var cached = await _cache.GetAsync($"user:{userId}");
    if (cached != null) return cached;
    // ...
}

// Correct for our system — no caching on PII endpoints
public async Task<UserData> GetUserAsync(string userId) {
    return await _repository.GetAsync(userId); // Always fresh for audit trail
}

2. Test Coverage Illusion

AI writes tests that pass but don’t cover edge cases the AI itself introduced. The coverage percentage looks good. The actual risk coverage is hollow.

3. Dependency Sprawl

AI solutions often introduce new packages when existing utilities would work. I’ve seen three PRs in one week add three different JSON serialization libraries when we have a standardized one.

4. Error Handling Gaps

AI handles the happy path well. Error handling is often pattern-matched from training data and doesn’t fit the actual error taxonomy of your system.

5. Silent State Mutations

The hardest to catch: AI code that modifies state as a side effect, especially in async contexts. It looks correct, the tests pass, and it breaks something three layers up.

The New Review Protocol: Four Questions Before You Merge

I’ve restructured our review process around four mandatory questions that didn’t exist before AI coding tools:

Q1: “What does this code assume about system state that it doesn’t verify?”

Every AI-generated PR, ask this explicitly. The AI assumes the database schema matches its training data. It assumes the API contract is what the docstring says. It assumes the environment variable exists.

Run this mental exercise: if every comment in this PR was removed, would a new engineer understand why this code does what it does, not just what it does?

Q2: “What happens to this code at 3x current load?”

AI-generated code often doesn’t account for concurrency, database connection pools, or rate limits. Force yourself to trace the async paths.

Q3: “What’s the blast radius if this fails silently?”

Silent failures are the AI’s specialty. Ask: if this returns a default value instead of throwing, what breaks downstream? What monitoring would catch it?

Q4: “Does this match how our team handles [pattern X]?”

Not how the internet handles it. Not how the framework documentation shows it. How your team, in your codebase, has decided to handle it.

New Metrics You Should Be Tracking Now

Stop measuring PR count. Start measuring:

Incidents-per-PR by Author-Type

Track separately: PRs from AI-heavy sessions vs. human-only sessions. Mine shows AI-heavy PRs have 2.1x higher incident rate on first week in production.

Review Time Efficiency

(Bugs caught in review) / (Hours spent reviewing) — this tells you if your review process is actually finding problems.

Time-to-Failure for Merged PRs

How quickly do incidents occur after merge? AI-generated code tends to pass initial QA then fail under production-specific conditions (specific data shapes, specific load patterns, geographic-specific issues).

CLAUDE.md Coverage Score

If you’re using Claude Code, track what percentage of your team’s repos have a complete CLAUDE.md with architectural constraints, naming conventions, and prohibited patterns. This is the highest-ROI input to AI code quality.

The Tooling Response: AI-Assisted Review

The irony isn’t lost on me: the solution to AI-generated code problems might be AI-assisted review.

We’ve been piloting two approaches:

Approach 1: Pre-commit Harness Review

Before a PR is even opened, a Claude Code /review skill runs against the diff and checks:

  • Does this match architectural patterns in ARCHITECTURE.md?
  • Are there any prohibited imports or patterns?
  • What’s the test coverage of the new code paths?

This catches ~40% of context-blindness issues before they ever reach a human reviewer.

Approach 2: Intent-Before-Code Reviews

Stolen from this Upstream article: the engineer writes a 3-sentence description of what they’re trying to accomplish before the AI generates code. The reviewer reviews the intent first, code second.

## Intent
Implement user preference caching to reduce DB calls on the dashboard.
CONSTRAINT: Must not cache PII fields (email, phone) per compliance policy #SEC-42.
EXPECTED BEHAVIOR: Cache TTL 15 minutes, bust on profile update.

This alone has cut our context-blindness incidents by 60% because the human constraint gets baked into the prompt.

What Actually Works: The 30-Day Plan

If you’re a Tech Lead reading this with AI tool adoption already underway, here’s the minimum viable response:

Week 1 — Measure Add [AI-assisted] label to PRs. Start tracking incidents separately. Get the baseline.

Week 2 — Constrain Write or improve CLAUDE.md in every active repo. Focus on: prohibited patterns, required conventions, compliance constraints, testing standards.

Week 3 — Process Roll out the intent-before-code template. Make it optional — you’ll get 50% adoption immediately and 80% within two weeks once engineers see it helps.

Week 4 — Review Look at your Week 1 baseline vs. current. Double down on what reduced incidents-per-PR.

The Uncomfortable Truth

AI coding tools are here to stay. The engineers who use them are more productive on raw output volume. The question is whether your review process has evolved to match.

The mistake I see most Tech Leads make is treating AI code review like fast human code review. It’s not. It requires different questions, different metrics, and a fundamentally different mental model of what you’re looking for.

The code is generated. Your job is verification, context-enforcement, and intent preservation. Those are human skills. They’re also, right now, the highest-leverage skills in engineering.

Have you changed your code review process for AI-generated code? I’d love to hear what’s working — find me on LinkedIn or GitHub.

Export for reading

Comments