The villa that upgraded itself

Picture spot-checking a migration batch: a language model extracting structured fields from legacy property listings. Record after record looks fine. Then one comes back changed. The source says plain "villa"; the output says "pool villa". No error. Valid JSON. Plausible — most villas in that market do have pools. And wrong: nothing in the source mentions a pool.

The model is not failing. It is helping. This is what LLM hallucination looks like in data extraction: invented detail that fits the pattern. That day the problem stopped being prompting and became verification. This post is the method I use to decide when an LLM task may become automation — and what it costs.

One reviewer, two pipelines, zero QA team

I work solo, inside SBX, a governance layer I built for agentic development, driving Claude Code and a Python agent stack. Two tasks were genuinely repetitive. The first: migrating roughly 86 legacy property listings into a new structured schema for a client's production real-estate platform in Thailand. The second: importing whole books into a structured knowledge base for my own workspace.

I am the only reviewer — no QA team. Any method that relies on a human catching mistakes scales only as far as my attention that day. Both now run as validated pipelines; this post is how they earned that.

Three standard answers, and the one that looked solved

Against "the model sometimes invents things", engineers reach for three reasonable tools.

Prompt-hardening. Explicit rules, few-shot examples (worked examples inside the prompt), "output only JSON, no commentary." I did this first, and it works: visible errors dropped fast, and after a day of tuning the output looked solved.

Provider structured output. Function calling or schema-constrained decoding guarantees parseable output; parse failures effectively disappear. If your problem is malformed output, this ends it.

Sampled human review. Read ten records and assume the rest match. Standard QA practice, fine for low-stakes batches.

My first theory: the first two tools were enough. Every metric I could see — parse failures, broken records — went to zero.

Why structured output alone missed the hallucination

  • Structured output validates shape, not truth. "Pool villa" and "villa" are both legal values for a property-type field. No schema knows which one the source supports. Valid syntax is not valid meaning.
  • Sampling misses plausible errors most of all. A truncated field jumps out at a reviewer; a believable upgrade does not.
  • Prompt-hardening decays. Rules tuned on one batch slipped on inputs the tuning never saw: longer listings, mixed-language fragments, a different model snapshot. Nothing announced the slippage; the failures were still well-formed.

The final clue came from elsewhere. I had instrumented my own agentic development process: the telemetry showed 82% of tasks force-closed instead of completing their checklists (tool-reported, my environment). Unenforced process decays — in humans and in agent loops alike. The same lesson drives my operating rules for AI coding agents: trust the mechanism, not the promise. A validation step that lives in a document, not in the execution path, will eventually not run.

So I stopped trying to make the model trustworthy and started making the loop refuse untrusted output. That is a trade-off, not a superiority claim: it costs real setup time, and it pays off only for tasks you run many times.

Prototype, schema, validator, hook — then call it established

The method, step by step:

  1. Prototype by hand. A handful of runs, enough to learn the task's real failure modes.
  2. Write the output schema — a formal definition of the fields and their allowed values.
  3. Write the validator — code that checks every record against the schema and the evidence rules.
  4. Wire validation into the loop so it cannot be skipped: a Claude Code hook (a check the tool runs automatically) in development, or an on-result-complete check in an MCP flow.

Validation is two-level: each record is checked against the schema, and the schema itself against schema-definition rules — schema drift is caught like data drift. Only when runs pass without me in the loop does the task earn the status "established" — and automation.

Each task is ETVX-shaped — Entry criteria, Task, Validation, eXit — an old IBM process pattern from the 1980s that fits agent work because it demands input and output validation:

task: refine-listing
entry:                      # validated before the model runs
  - listing.source_text present and non-empty
task_body: prompt.md        # definition, steps, examples, references
validation:                 # guides the verifier, not just pass/fail
  - schema: listing.v2.schema.json
  - rule: every structured value traces to evidence; unknown => null
exit:
  - validator passed; report archived

The validation section guides the verifier: what to check, in what order — not just yes or no. The task body is a prompt-engineered document, and each task is assigned to an agent with a role and a personality — also prompt-engineered fields, not decoration.

Composing these documents by hand became unmanageable, so I built Prompt Studio (promptstudio.shredbx.com). Common blocks become presets — edit the preset, every consumer re-resolves — with templates per prompt type and variables filled at run time. I describe it in prompt management as linked documents, not strings. Presets, templates, variables, and multi-format rendering run today; the connector layer is designed, not built.

Two pipelines run under it. The book importer: a chapter-analyzer agent and a knowledge-extractor agent place entries into a 264-cell capability × artifact × process-layer matrix. The listing refinement: five ordered skills — extract, clean, rewrite, validate, translate to Thai — over the ~86 listings. One rule covers both: every structured value must trace to evidence in the source; unknown stays null. The villa stays a villa.

I will not synthesize a before/after error rate — I never measured the unvalidated baseline. What I can say honestly: evidence-rule violations are now caught mechanically, per record, instead of by sampled human review. And the cost is real. For a small task the validator is often more work than the prompt. It also adds a new failure mode: a wrong validator silently rejecting good records.

If your errors look plausible, this is your problem too

The general shape: a many-record LLM transformation where wrong outputs are valid in form and believable in content. That covers more than property listings — contract and invoice field extraction, support-ticket triage into structured tags, localization batches, CRM enrichment. If a wrong record would embarrass you weeks later rather than crash anything today, you are in this territory.

Who should not do this: one-off explorations; tasks whose output a human reads end-to-end anyway (prose drafts); tasks whose schema would change weekly. There, prompting plus spot checks is the right cost/benefit; a validator would be ceremony.

Still ugly: hand-written validators, and a pipeline layer that is a drawing

The honest gaps. Semantic validators are hand-written per task; the schema-checks-the-schema trick catches structural drift, but the evidence rules are still custom code. There is no statistical evaluation layer — my checks are deterministic; sampling-based quality evals are the obvious next tier. And the Prompt Studio pipeline layer — connectors and transformers between tasks — is a design, not a build. Today I compose the pipelines, not the tool.

The next version is clear: n8n-style graphs of ETVX tasks, each hop validated on entry and exit. Starting today, I would change one thing: begin the validator the same day as the prompt, not after the prototype feels done. "Feels done" was the signal that lied to me.

Takeaways

  1. The dangerous LLM errors are plausible, not malformed. The pool villa passed every syntactic check; validate meaning against source evidence.
  2. A check not wired into the loop will eventually not run. Documents do not enforce; hooks and on-complete gates do.
  3. "Unknown stays null" beats "best guess." Nulls are recoverable; confident inventions are not.
  4. Treat task definitions as engineered artifacts. ETVX turns a prompt into a process step you can compose and audit.
  5. Verify before you compose. "Established" is a status a task earns, never a default.

Where this lives

The method runs inside my workspace tooling (SBX) and Prompt Studio (promptstudio.shredbx.com); notes and future posts at shredbx.com.

External references

Related projects