"Done — all tests pass." Nothing had run.

The transcript said "Done — all tests pass." I scrolled up to find the test output. There was none. The agent had not run the suite. It had predicted the sentence that usually follows a finished task.

That same week, I re-explained — for the fourth time — where the entity definitions live, which wrapper runs the tests, and what "done" means in this repo. The assistant had been told all of it the day before.

This post is about the fix I built: a development lifecycle (SDLC) that agents walk step by step, with machine-checked gates. And about what measuring it revealed — 82% of my tasks were being force-closed: ended by my own override, not by passing the gates.

One engineer, 110 packages, and a brilliant assistant with amnesia

I work solo in a monorepo that spans, by the tooling's own count, roughly 110 packages and 16 applications — Go services, SvelteKit frontends, Swift, Python. One of them is a production real-estate platform I build for a client in Thailand. Claude Code does most of the implementation work.

The constraint that shaped everything: there is no second human. Nobody reviews the agent's claim that tests passed. A fabricated "done" ships.

So I did not need a smarter agent. I needed a process in which a false "done" is mechanically impossible to record. SBX, the framework this post describes, is my attempt at that property. It runs my daily work; it is not a product.

The standard playbook: write it all down, bigger

The obvious move — the one I made first — is the instruction file: CLAUDE.md, a system prompt, "project rules." Every correction becomes a paragraph: where files live, which commands to run, what done means. It works, at first. It is cheap, versioned with the repo, and the vendor guidance for it is genuinely good.

The second option is per-task templates and checklists — a definition of done pasted into every task. The third is memory and retrieval: store conventions and past decisions somewhere structured, and pull in only the relevant ones per task.

All three do real work; the instruction file alone carried me for months. But they share one assumption I did not examine until it failed: that the agent will follow the instructions it can see.

Instructions persuade. They don't enforce.

Where it broke, in order.

The instruction file drifted and bloated. Each failure became another bolded rule. Every token taxed every task, and the more rules the file carried, the less weight any one rule had.

The "done" incident above broke my mental model. The file already contained a rule — in bold — that success claims must quote the verifying command output. The agent had read it that session. It still wrote the sentence without the output. A language model is a probabilistic reader, not a rule engine.

Checklists failed one layer up: the agent marks its own checkboxes. That is self-graded homework. Retrieval helped with knowledge but stayed best-effort — the wrong artifact at the wrong step is the same as no artifact.

The reframe that mattered: I did not have a knowledge problem. I had an enforcement problem. Knowledge problems are solved by writing things down. Enforcement problems are solved by mechanisms that do not care what the text says — permissions, gates, captured evidence. Many of these mechanisms began as the operating rules I collected from running AI coding agents daily. It is a trade-off: mechanism is expensive and rigid where prose is cheap and flexible. My constraint — no human reviewer between the agent and production — priced prose out.

What I built: the SDLC as a walked process

In SBX, the lifecycle is data, not habit. A task walks through 25 step templates — requirements, modeling, tests, implementation, review, close — declared in YAML. Each step has entry and exit gates: the ETVX shape (entry, task, validation, exit) that IBM process engineering described in 1985.

Each step declares:

  • what it requires and what it produces;
  • which of 5 role-typed agents runs it — the reviewer role is read-only simply because it is never given write tools;
  • which knowledge artifacts get injected into the prompt at that step — from a library of 231 (52 rules, 143 guidelines, 36 smells, tool-reported). Guidance arrives at the moment it applies, not as a preamble tax.

Around the walk sit the hard mechanisms. A pre-write hook rejects edits outside the task's declared scope. A command deny-list blocks raw build and git commands, so agents act only through sanctioned wrappers. And "proof-by-command" captures the stdout, stderr, and exit code of build, test, and lint runs as structured JSON evidence attached to the gate. A "done" without evidence is not disbelieved — it is unrecordable.

# step contract (abridged)
step: implementation
entry:    [tests_defined, scope_declared]
produces: [code_changes, build_evidence, test_evidence]
agent:    implementer        # writes only inside declared scope
inject:   [coding-standard, error-handling-guideline]
exit:
  gate: proof-by-command     # build/test output captured as JSON

Then I measured it: 82% force-closed

The walk engine logs everything, so I analyzed my own runtime corpus. Measured, in my case: roughly 82% of tasks were force-closed — ended by override, not by passing the final gates. Planning steps had a median dwell time (time spent on the step) under one minute. That is paste-through, not planning.

My carefully gated front half was theater. The real planning happened in discussion, before the task ever existed. So I redesigned around the data: a plan produced in that discussion is handed to task initialization and auto-satisfies the early gates. The walk now starts at implementation, where enforcement actually earns its cost. That plan artifact is one piece of a larger bet — the spec as the single source of truth that AI agents read at every task start.

The system has governed about 436 task runs so far (tool-reported). The bill is real too. The engine lives inside a roughly 270k-line Go CLI (tool-reported) that I alone maintain. The scope hook sometimes blocks legitimate writes mid-flow. Every gate is a tax I have to keep re-justifying.

Where this transfers: spec-driven development, enforced

The general shape: a capable but unreliable executor, working inside an environment that can verify its work. If that describes your team, the transferable move is shifting rules from prose to mechanism. It is the enforcement half of what the industry now calls spec-driven development: the plan drives the work, and gates verify the work actually happened.

Concretely:

  • require agent-authored PRs to carry captured command evidence, not claimed results;
  • enforce write scope with permissions or sandboxes, not instructions;
  • inject standards at the step where they apply, not into one giant prompt.

The same shape shows up in CI policy for agent-generated changes, and in platform teams building paved roads for internal agents. Who should not do this: anyone prototyping, anyone running a handful of agent tasks a week, and any team whose human code review is already tight. There, the instruction file remains the right tool.

What's still ugly

The gates verify that commands ran and exited zero — not that the tests mean anything. An agent that writes weak tests passes honestly. Force-close still exists by design. The 82% figure is a warning: when gates outnumber their value, overrides become the norm. Every number here comes from my own single-user corpus, tool-reported, not a controlled study.

And the engine is bespoke. Starting today, I would build the same properties on standard primitives: the hook and permission systems agent CLIs now ship with, plus MCP (Model Context Protocol) for tool surfaces. Custom code would remain only for the walk itself and the evidence capture. The next level is this contract applied to a team: shared gates, per-role permissions, and evidence as the unit of code review.

Takeaways

  1. Instructions persuade; mechanisms enforce. If breaking a rule must be impossible, it needs a hook or a permission, not a paragraph.
  2. Capture evidence, not assertions. A done-claim backed by structured command output turns an agent's word into something checkable — on any stack, with any model.
  3. Measure your process before hardening it further. The 82% force-close rate disqualified half of my own design. Without the data, I would have defended it indefinitely.
  4. Deliver knowledge at the step where it applies — 231 artifacts injected per step, instead of one bloated preamble competing for attention.
  5. Treat scope as a permission, not a convention. An agent that cannot write outside its task cannot drift outside it either.

Where the code lives

SBX runs my daily work behind shredbx.com; it's a personal framework, not something you can install today. If you take one thing: pick the rule you currently write in bold most often, and build the mechanism version of it.

External references

  • Best practices for Claude Code — the instruction-file playbook steelmanned above, and worth doing well before building anything heavier.
  • Building agents with the Claude Agent SDK — Anthropic's engineering guidance on agent loops, tools, and permissions.
  • Radice et al., "A Programming Process Architecture," IBM Systems Journal 24(2), 1985 — origin of the ETVX (entry–task–validation–exit) step pattern; summarized in this software process kernels overview.
  • DORA — the research program connecting delivery-process capabilities to outcomes; useful framing for measuring a process before redesigning it.

Related projects