Most automation failures don’t start with bad code — they start with bad planning. I’ve watched teams automate the wrong things, pick the wrong tools, and burn out within two months. Not because automation is hard, but because they skipped the planning step entirely.

This is Part 2 of our series. Before you write a single test, you need a strategy. This post gives you a practical framework for planning automation regardless of what kind of project you’re working on.

The Automation Decision Matrix

Not everything should be automated. This is the most important lesson I’ve learned: automate strategically, not comprehensively. Use this matrix to decide what goes first:

CriteriaScore 1 (Low)Score 5 (High)
FrequencyRun once a quarterRun every sprint/day
StabilityFeature changes every sprintFeature is stable for months
CriticalityNice-to-have featureRevenue-impacting / core flow
ComplexitySimple one-step checkMulti-step flow with many states
Manual TimeTakes 30 seconds manuallyTakes 15+ minutes manually

Score each test case on these five criteria. Automate the highest-scoring tests first. A test that’s run daily, on a stable feature, for a critical flow, with complex multi-step verification, and takes 15 minutes manually — that’s a score of 25/25. Automate it today.

What to Automate vs. Keep Manual

Always automate:

  • Login and authentication flows
  • Core business transactions (checkout, orders, payments)
  • Data validation and form submissions
  • API health checks and contract tests
  • Smoke tests for critical pages
  • Regression tests for stable features

Keep manual (for now):

  • Brand new features still in active development
  • One-time data migration verification
  • Complex visual design reviews
  • Usability and accessibility assessments
  • Exploratory testing of edge cases
  • Tests requiring physical device interaction

Consider carefully:

  • Features that change frequently (high maintenance cost)
  • Tests requiring complex test data setup
  • Integration with third-party systems you don’t control
  • Tests that need human judgment (“Does this look right?”)

Planning for Different Project Types

Every project type has different automation priorities. Here’s how to approach each:

Web Applications (SPA / SSR / Static Sites)

Web apps are the sweet spot for Playwright. You get the most value fastest.

Priority order:

  1. Smoke tests — Does the app load? Can users navigate? Do critical pages render?
  2. Authentication flows — Login, logout, session handling, password reset
  3. Core business flows — The 3-5 most important user journeys (e.g., search → view → add to cart → checkout)
  4. Form validation — Submit with valid data, invalid data, empty fields, special characters
  5. Responsive testing — Same tests on mobile viewport and desktop
  6. Visual regression — Screenshot comparison for key pages

Example test roadmap for an e-commerce web app:

Week 1: Homepage loads, product listing renders, search works
Week 2: Login, registration, password reset
Week 3: Product detail → add to cart → checkout → confirmation
Week 4: Filters, sorting, pagination, empty states
Week 5: Mobile viewport tests for weeks 1-4
Week 6: Visual regression baselines, error state tests

Tools: Playwright (primary), Playwright MCP with Claude (test generation)

API / Microservices

API testing is faster to automate than UI testing because there’s no browser overhead. If your project is primarily API-based, start here.

Priority order:

  1. Contract tests — Does each endpoint return the expected shape?
  2. Status code verification — 200 for valid requests, 400 for bad input, 401 for unauthorized
  3. Data validation — Response fields match expected types, required fields present
  4. Error handling — Invalid payloads, missing fields, boundary values
  5. Integration tests — Multiple endpoints called in sequence (real user flow)
  6. Performance baselines — Response time under load

Example test structure:

test.describe('Products API', () => {
  test('GET /api/products returns product list', async ({ request }) => {
    const response = await request.get('/api/products');
    expect(response.ok()).toBeTruthy();

    const data = await response.json();
    expect(data.products).toBeInstanceOf(Array);
    expect(data.products.length).toBeGreaterThan(0);
    expect(data.products[0]).toHaveProperty('id');
    expect(data.products[0]).toHaveProperty('name');
    expect(data.products[0]).toHaveProperty('price');
  });

  test('GET /api/products/:id returns single product', async ({ request }) => {
    const response = await request.get('/api/products/1');
    expect(response.ok()).toBeTruthy();

    const product = await response.json();
    expect(product.id).toBe(1);
    expect(product.name).toBeTruthy();
  });

  test('POST /api/products rejects invalid data', async ({ request }) => {
    const response = await request.post('/api/products', {
      data: { name: '' }, // Missing required fields
    });
    expect(response.status()).toBe(400);
  });
});

Tools: Playwright request fixture (no browser needed), Postman for exploration

Mobile Applications

Mobile testing is more complex due to device fragmentation. For QC teams starting out, I recommend a pragmatic approach.

Strategy:

  1. Test the mobile web version first — Use Playwright’s device emulation
  2. API tests cover the backend — Same as the API section above
  3. Native mobile testing later — Appium or Detox when the team is more experienced

Playwright device emulation gives you a lot for free:

import { devices } from '@playwright/test';

// In playwright.config.ts
projects: [
  {
    name: 'Mobile Chrome',
    use: { ...devices['Pixel 7'] },
  },
  {
    name: 'Mobile Safari',
    use: { ...devices['iPhone 15'] },
  },
],

This runs your existing tests on mobile viewports. You’ll catch responsive layout issues, touch target size problems, and mobile-specific bugs without any native mobile tooling.

Legacy Systems

Legacy systems are the trickiest because the UI is often inconsistent, there’s no test environment, and the code has zero tests.

Approach:

  1. Characterization tests first — Tests that document current behavior, not expected behavior
  2. Record and replay — Use Playwright’s code generation to record manual flows
  3. Focus on regression — If you can’t add features easily, at least prevent regressions
  4. API-level tests if possible — Bypass the ugly UI and test the backend directly

Playwright Codegen is perfect for legacy systems:

npx playwright codegen http://your-legacy-app.com

This opens a browser and records every action you take, generating Playwright test code automatically. Even if the generated code needs cleanup, it’s a fast way to document existing flows.

Creating Your Automation Roadmap

Here’s a template for presenting your automation plan to your team and stakeholders:

Phase 1: Foundation (Weeks 1-2)

Goal: Prove that automation works and delivers value.

TaskOwnerTime
Set up Playwright projectQC team2 hours
Write 5 smoke tests for critical pathsQC team3 days
Set up CI pipeline (GitHub Actions)QC + Dev1 day
Demo automated tests to stakeholdersQC team1 hour

Success metric: 5 automated smoke tests run on every PR.

Phase 2: Coverage (Weeks 3-6)

Goal: Cover the top 20 most-tested regression scenarios.

TaskOwnerTime
Create Page Object classes for key pagesQC team1 week
Automate top 20 regression test casesQC team2 weeks
Add API endpoint testsQC + Dev1 week
Set up test reporting (HTML reports)QC team2 hours

Success metric: 25+ tests, 80%+ pass rate in CI, regression testing time reduced by 50%.

Phase 3: AI-Assisted Expansion (Weeks 7-10)

Goal: Use AI tools to triple test coverage.

TaskOwnerTime
Set up Claude/Copilot for test generationQC team1 day
Generate tests for remaining user flowsQC team + AI2 weeks
Add visual regression testsQC team3 days
BDD feature files for critical flowsQC + PO1 week

Success metric: 60+ tests, AI generating first drafts of most tests, visual regression catching CSS issues.

Phase 4: Maturity (Weeks 11-14)

Goal: Self-sustaining automation practice.

TaskOwnerTime
Cross-browser testing (Chrome, Firefox, Safari)QC team3 days
Performance testing baselinesQC + Dev1 week
Team training on test maintenanceQC team2 sessions
Establish Definition of Done with automationQC + Dev + PO1 meeting

Success metric: Every new feature includes automated tests. Manual regression is a 30-minute review, not a 4-hour grind.

Estimating Automation ROI

Stakeholders want numbers. Here’s how to calculate them:

Time Savings

Manual regression time per release:     4 hours
Number of releases per month:           4
Total manual testing per month:         16 hours

After automation:
Automated test run time:                15 minutes
Manual exploratory time per release:    1 hour
Total testing time per month:           5 hours

Time saved per month:                   11 hours
Time saved per year:                    132 hours

Cost Savings

QC team hourly cost (blended):          $40/hour
Annual time saved:                      132 hours
Annual cost savings:                    $5,280

One-time setup cost:                    ~80 hours × $40 = $3,200
Ongoing maintenance per month:          ~5 hours × $40 = $200

Break-even point:                       ~4 months

Quality Improvement (harder to quantify but real)

  • Faster feedback — Bugs caught in minutes, not days
  • Consistency — Every test runs the same way every time
  • Confidence — Release with evidence, not hope
  • Coverage — Tests run 24/7, not just during business hours
  • Regression prevention — Automated tests don’t forget to check yesterday’s fix

Choosing the Right Tools

For QC teams starting automation in 2026, here’s my recommendation:

NeedToolWhy
UI TestingPlaywrightBest locators, auto-waiting, great debugging, Microsoft-backed
API TestingPlaywright request fixtureSame tool, no context-switching, built-in assertions
BDDCucumber + PlaywrightGherkin syntax lets QC write scenarios in English
AI Test GenerationClaude Code + Playwright MCPAI explores the live app and writes tests from accessibility tree
AI Code CompletionGitHub CopilotInline suggestions as you type test code
AI Agentic WorkAntigravityAutonomous test generation and debugging
CI/CDGitHub ActionsFree for public repos, integrates with everything
ReportingPlaywright HTML ReporterBuilt in, no extra setup
Visual RegressionPlaywright toHaveScreenshot()Built in, no external service needed

Why Playwright over Selenium, Cypress, or TestCafe?

  • Auto-waiting — No more sleep(3000) to wait for elements. Playwright waits automatically.
  • Multiple browsers — Chrome, Firefox, Safari from one script
  • Device emulation — Mobile testing without physical devices
  • Network interception — Mock APIs without backend changes
  • Trace viewer — Time-travel debugging for CI failures
  • MCP integration — AI can control a real browser and generate tests

Common Planning Mistakes

Mistake 1: Automating Everything at Once

Symptom: The team tries to automate 200 test cases in month one. Result: Burnout, flaky tests, abandoned project. Fix: Start with 5-10 high-value tests. Get them green and stable. Then expand.

Mistake 2: Automating Unstable Features

Symptom: Tests break every sprint because the feature keeps changing. Result: More time maintaining tests than running them. Fix: Only automate features that are stable. Use manual testing for features in active development.

Mistake 3: No Maintenance Plan

Symptom: Tests are written but nobody updates them when the app changes. Result: Suite becomes unreliable, team loses trust, tests get disabled. Fix: Budget 20-30% of automation time for maintenance. Include test failures in your sprint board.

Mistake 4: Not Involving Developers

Symptom: QC team writes tests in isolation. Dev team doesn’t know or care. Result: No data-testid attributes, no API stubs, no test environment support. Fix: Include developers from day one. They add data-testid attributes. You write the tests. Everyone reviews.

Mistake 5: Picking the Wrong Metric

Symptom: Team celebrates “200 automated tests!” but all 200 test the same page. Result: False confidence. Critical paths are untested. Fix: Track scenario coverage, not test count. Cover the most important user journeys first.

Your Automation Planning Template

Here’s a one-page template you can fill out for your project:

# Automation Plan: [Project Name]

## Scope
- Application type: [Web / API / Mobile / Legacy]
- Current test approach: [Manual checklist / Ad-hoc / None]
- Releases per month: [Number]
- Manual regression time: [Hours per release]

## Priority Tests (Top 10)
1. [Test scenario] — Score: [X/25]
2. [Test scenario] — Score: [X/25]
...

## Timeline
- Phase 1 (Weeks 1-2): Foundation — [X] smoke tests
- Phase 2 (Weeks 3-6): Coverage — [X] regression tests
- Phase 3 (Weeks 7-10): AI-Assisted Expansion
- Phase 4 (Weeks 11-14): Maturity

## Tools
- Testing: Playwright
- AI: Claude / Copilot / Antigravity
- CI/CD: GitHub Actions
- BDD: Cucumber (if applicable)

## Expected ROI
- Monthly time saved: [X] hours
- Break-even: [X] months
- Quality improvement: [Qualitative description]

## Team
- QC owner: [Name]
- Dev support: [Name]
- Stakeholder: [Name]

Series Navigation

In Part 3, I’ll take you through your very first Playwright test — step by step, assuming zero coding experience. You’ll install everything, write a real test, and see it pass. No prior knowledge required.

Export for reading

Comments