IDEAS, DECISIONS & LESSONS

Journal

Notes on AI, architecture and building useful products.

917 articles

26 Aug 2026 · 4 min read

agentic engineering

You Can't Ship Agents on Vibes — Evals for Agentic Systems

A one-line prompt tweak can silently break an agent, and you won't notice until a user does. My fix is an eval harness: a golden set of inputs and checks, run against every agent and workflow on every change. Here's the anatomy — deterministic checks, LLM-as-judge, run-on-change regression, and grading multi-agent workflows.

Read more