On September 11, 2026, Simon Willison shipped Datasette 1.0a39 and 0.65.4 — security releases closing permission-check gaps that let authenticated users on public Datasette instances see data they shouldn’t have been able to reach. The advisory is deliberately terse on specifics (“if you are running Datasette instances on the public internet you should upgrade now”), which is the right call for a live vulnerability disclosure. What’s more interesting to a technical lead than the patch itself is how the audit that found these bugs was run, because it’s a concrete, dated example of an AI-assisted security workflow that isn’t “paste the codebase into a chatbot and hope.”

The trigger and the process

The audit wasn’t proactive housekeeping — it started because security researcher Sevban Dönmez sent Willison several AI-assisted vulnerability reports. Rather than patch those in isolation, Willison and fellow maintainer Alex Garcia used the reports as the seed for a broader, systematic review, run privately over roughly a week before the public release.

The workflow, in Willison’s own description, was deliberately structured to keep humans in the loop on both ends of each fix:

“One of us would create the automated tests highlighting the issue, then the other would implement the fix.”

That single sentence is the whole design pattern worth stealing. Two humans, three frontier models (Claude Fable 5.1, GPT-5.6 Sol, and GPT-6 Astra) doing the pattern-matching and hypothesis generation, and a hard rule that no fix ships without a second human independently reviewing it against a test that demonstrates the bug. Willison notes the team found “some very subtle bugs” — the kind that needed frontier-model-assisted pattern matching across the codebase to surface at all, not the kind a human reviewer would spot skimming a diff.

Why the two-person handoff matters more than the model choice

It’s tempting to read this story as “which model found the bug” — that’s the wrong axis. The valuable part is the handoff discipline: person A writes a failing test that proves the vulnerability exists, independent of any fix; person B writes the fix against that test, without necessarily having written the original report themselves. This breaks the most common failure mode of AI-assisted security work, which is a single person using a single model to both find and fix an issue with no adversarial check in between — a setup where the model’s blind spots and the human’s blind spots can silently overlap.

Compare this to the sloppier pattern many teams default to: “ask the coding agent to review the auth code for bugs,” accept whatever patch it proposes, ship it. That workflow has no test-first proof the bug is real, no second reviewer, and no separation between discovery and remediation. Datasette’s process forces a falsifiable claim (the failing test) before any code changes, which is standard TDD discipline applied specifically to security findings — arguably the highest-leverage place to apply it, since a wrong assumption about a vulnerability’s shape is exactly the kind of mistake that ships a fix which doesn’t actually close the hole.

A hands-on pattern you can lift this week

Here’s how to adapt this for your own stack, without needing three frontier models or a maintainer team:

# 1. Seed the audit from a real signal, not a blank prompt.
#    A user report, a fuzzer finding, a pentest note — something concrete.
git checkout -b security/audit-permission-checks

# 2. Person/session A: write a failing test that encodes the suspected bug.
#    Ask the model to help enumerate attack surface, but YOU write the assertion.
cat > tests/test_permission_regression.py <<'EOF'
def test_authenticated_user_cannot_read_other_users_private_table(client):
    # Arrange: user A owns a private table, user B is authenticated but unrelated
    resp = client.get("/private-db/user-a-table.json", auth=user_b_token)
    assert resp.status_code == 403  # currently fails — returns 200
EOF
pytest tests/test_permission_regression.py  # confirm RED

# 3. Person/session B (different context, ideally a different reviewer):
#    implement the fix against the test, with no visibility into how A phrased the bug report.
#    Only success criterion: the test above goes GREEN, and no existing tests go RED.
pytest tests/  # confirm full suite GREEN

# 4. Ship only with both the regression test AND the fix in the same PR,
#    reviewed by someone who did not write the fix.

The mechanical trick is step 3’s isolation: whoever implements the fix should ideally not see the exact wording of the original vulnerability report, only the failing test. That forces the fix to satisfy the actual security property being tested, not just pattern-match the reporter’s mental model of the bug — which is often incomplete in exactly the way that leaves a variant of the same hole open.

What this means for your team’s security backlog

If your org treats “have the AI review this for security bugs” as a checkbox that happens once before a release, you’re getting a fraction of the value Datasette got here. The three things worth copying directly:

  • Seed audits from real reports, not scheduled busywork. Dönmez’s reports gave the team a concrete starting shape; an unguided “scan the whole codebase” prompt tends to produce noisy, low-signal output because the model has no prior on what’s actually exploitable in your specific deployment topology.
  • Separate the discovery session from the fix session, even if it’s the same human switching hats — clear the model context, restate only the test, not the original theory of the bug.
  • Gate every fix behind a red-to-green test, and disclose on your own timeline. Willison’s team held the automated tests back from the public post specifically to give downstream users an upgrade window before the exploit shape became public knowledge — a good default for any vulnerability affecting self-hosted software with a long tail of unpatched instances.

If you maintain any self-hosted tool with an admin/permissions surface — internal dashboards, BI tools, anything with row-level access control — this is a good week to run the same exercise, even without three frontier models on staff. The test-first, two-reviewer discipline is the transferable part; the specific models are a footnote.

Sources: Datasette security releases announcement, Simon Willison’s write-up

Export for reading

Comments