Notes · AI coding agents · Multi-agent systems

One Mind to Build, One Blind Mind to Break

November 17, 2026 (updated Nov 18 with a 4-pair follow-up replay) · ~2,800 words · 4 interactive figures and a quiz · Prefer the scientific paper format? Read Paper 1 & Paper 2 (HTML, PDF & Markdown) · AI agent? read agents.md

TL;DR

Does splitting a coding agent into isolated "maker-checker" roles beat a single-context coding session? Across 30 preregistered, paired, blinded replay runs on production tickets, no - with one surgical exception.

  • Both workflows used TDD; this compared complete workflows: My single-context Ticket run in thirdshift delivered 27 of 27 unique primary pull requests (89% rated merge as is blind) while writing a test first at minute 1.3–1.8. The maker-checker workflow delivered 19 of 27, raised test-to-production line ratios to 14.7:1 and 10.10:1, and cost $422.84 and about 49.5 active hours across the 30 pairs vs. $39.32 and 8.0 hours, summing elapsed trial time in both studies. Study 1 also differed in orchestrator and review configuration; Study 2 equalized the top-level orchestrator while retaining other workflow differences.
  • Equalizing the orchestrator halved cost, not failure modes: Moving the multi-agent orchestrator from Claude Code to Codex in Study 2 dropped the median cost ratio from 10.4× to 4.62×, yet the blind maintainer still preferred the single-agent run 8–0–3 (and a blind colleague scored 1–0–1 on two sampled pairs), while a 17-line web server access-control rule trapped the multi-agent pipeline in an 11-hour 39-minute, 64-session, $64.22 loop with no PR opened.
  • The one mechanism worth keeping: Post-hoc, diff-blind behavioral execution exposed real bugs in 2 of 19 delivered pairs (Study 1 Pairs 6 and 7) that single sessions and diff reviewers missed - falling to 0 of 11 in Study 2. Instead of pre-code handoffs, I am testing deterministic tests_on_base checks on every ticket and a selective, runner-bounded +1 Blind Behavioral Breaker fed only public signatures (public_api_delta.txt); an initial 4-pair follow-up replay cut false positives to 0/4 at ~$0.32 API per gated task, caught an unguarded zero-date variant in Study 1 Pair 7, and missed Pair 6's deeper inline-signature bug within 139 test lines.

Prefer the scientific paper format?

This essay synthesizes two preregistered, blinded experimental papers written in full academic format with methods, per-task tables, statistical tests, and typeset PDFs:

Separating coding agents into isolated planner, test-writer, implementer, and multi-round reviewer roles degraded delivery, multiplied cost by 4.6× to 10.4× across 30 paired replay runs, and produced half as many merge-ready pull requests as one continuous coding session - yet executing one independently written behavioral test suite after the fact caught real defects that a single session and its reviewers missed.

I ran these experiments because I carry the merge gate. At Educational Travel Adventures, most bug fixes and small features on Tourbot - our million-line PHP 8 and MySQL 8 operations platform - start inside thirdshift, an open-source Rust orchestrator I built to run coding agents cleanly. One Codex session reads an issue, writes a failing test first, builds the change, spawns parallel spec, standards, and security sub-agents, and opens a pull request. That ships fast, but it breaks a rule from Cleanroom software engineering (Mills, Dyer, & Linger, 1987): nobody should grade their own homework. If one context misreads a ticket, both its code and its test suite carry the blind spot. Before migrating our upcoming feature builds onto an isolated multi-agent Maker-Checker TDD pipeline, I wanted to measure what role separation actually buys - and what it quietly breaks.

Each section below starts folded so the questions read as the argument in brief. Click any heading to open it, search natively with Find in Page, or .

Synthesized from our two internal preregistered experiment reports (30 paired Tourbot replays), session traces, and primary literature notes, with AI drafting assistance. Figures 1, 2, and 4 use hand-built inline SVG rendered by local JavaScript; Figure 3 switches hand-built DOM cards. All four include static text fallbacks. Every metric, blind human call, and meta-reviewer trace quote traces directly to Research Paper 1 and Research Paper 2; all architecture decisions and open questions are mine.

For AI agents: agents.md contains the workflow contract, verified numbers, operational checklist, and self-test with answers.