A method for governing what counts as done

The work

New to EDD? Start with Make Done Demonstrable. The plain-language explainer shows why EDD is acceptance governance rather than merely more testing, then walks through the protected expectation, fail-closed evidence, codebase integration, and maintenance cost. Read the field report only with the audit correction on its card below; read the paper for the complete method, qualifications, and acceptance criteria.

In plain terms

AI coding agents can produce working code faster than teams can carefully check it. Generation can accelerate more quickly than review capacity, so work accumulates at the verification boundary. What you can delegate is set by what you can verify.

The usual answers help, and none of them finish the job. Tests the agent writes inherit the agent’s understanding, including the part that was wrong; they are a second expression of one opinion, not a second opinion. Code review is strong on what was written badly and weak on what was never written at all: the field nobody carried through, the record nobody wrote. Code review also becomes more expensive as the volume of generated code grows. Trust cannot replace verification either: METR has documented frontier models modifying tests or scoring code, accessing reference implementations or answers, and exploiting other loopholes to earn higher scores without solving the assigned task.

The idea. The central mechanism is Gold-State Parity™. Many business systems already contain an overlooked testing asset: trusted records in the exact shape the application is supposed to produce. EDD calls this the gold state and uses each record twice. First, it becomes the question—the input projection: values supplied through the declared application entry path, such as names, amounts, and selections; internal identifiers, audit history, and other system-generated values are withheld. Second, it becomes the answer key—the expected-state projection: the contract-complete persisted outcome at the comparison point, including derived values, audit rows, queued notifications, and linked child records. Run the question through the real application, then compare everything it stored with the answer key.

Isn’t that circular? Only if the application creates or can access its own answer key. It cannot. The gold state is created independently, versioned, protected, and unavailable to the application under test. Each run begins from a declared baseline containing none of the scenario’s business data, supplies only the input side, and requires the application to construct the result itself. The run answers one question: did the application reproduce the approved gold state? Whether that gold state reflects what the business actually wants is a separate governance question, settled before the run by human approval. The method keeps those questions separate and never allows the application to certify its own reference.

Matching means all of it. Matching row counts is not enough. Every in-scope table, record, field, and durable side effect must be accounted for. Each element must be:

• compared exactly;
• translated by a declared mapping when internal identifiers legitimately differ;
• verified by an explicit rule, such as a header total equaling the sum of its lines; or
• excluded for a documented reason and covered by a separate gate.

Anything that fits none of those categories fails the gate. Add an unclassified column and the comparison remains red until a human determines how it should be handled. Unknown state is never ignored or presumed harmless. That fail-closed rule is what makes the check difficult to weaken quietly.

Two paths, because full-interface verification does not scale. Driving a multimillion-record corpus through a real browser could take hours, days, or longer—long enough to make exhaustive interface-driven verification operationally impractical. EDD therefore divides the gate into two paths: Prove and Validate. The Prove lane runs a small, deliberately selected set of scenarios through the actual screens, with no shortcuts or test-only back doors. This verifies how supported user actions are translated into application instructions and records their exact shapes. The full corpus is then validated through the application’s normal boundary, using only instruction shapes previously observed at the interface. Any case that matches no interface-demonstrated shape fails rather than being inferred. The Prove lane verifies the translation from screen to application; Validate verifies the underlying write path across the entire in-scope corpus.

When it fails. The gate does not report “something broke.” It identifies the scenario, record, business key, field, step, expected value, and actual value. That produces an actionable fact rather than an impression: a single artifact that a person, an assisted diagnosis, or an unattended agent can consume. This precision makes delegating the repair to an agent a governed act rather than an act of faith.

The agent cannot alter the standard against which it is measured. The oracle, its scenarios, the gold state, and the recorded command evidence are cryptographically identified and remain outside the agent’s write authority. Every rerun revalidates the starting state. The repair loop can change the code, but not the oracle, gold state, or command evidence. Nothing passes until the failed scenario is rerun and the stored result is verified. Pass or fail is determined by declared rules and recorded evidence, not by the agent’s judgment.

One failure, end to end. In a fault-injection exercise, the operator deliberately planted a defect without revealing it to the repair machinery. A write path stored one valid identifier in the field intended for a related identifier. The value existed and had the correct type, but it was written to the wrong field. Everything that merely read it back reported success: the record loaded, the screen rendered, and a read-back test would have found a plausible identifier. The defect surfaced only when the application rebuilt a request from its own stored state. Reused in its new role, the corrupted value matched no valid baseline. The runner recorded the cycle:

RESULT: FAIL — expected a second version; the version strip still shows v1

  Iteration 2/5 | Type: API contract
  Hypothesis: the assembler persists the outer selector’s identifier into
              the inner selector’s field. The first submission succeeds and
              stores the corrupted value; the console resumes from the stored
              record, so the second submission carries an inner selector equal
              to the outer one and matches no template baseline.
  Evidence:   the second submission was refused outright; its request body
              carries the outer identifier in the inner field; the earlier
              options call on the same run shows the true inner value.

-- fix applied; driver relaunched at the failed scenario --
RESULT: PASS (all 35 of the scenario’s tests; fix witnessed)

One column. One valid identifier in the wrong field. A read-back check could confirm that the system stored what it wrote; it could not establish that the system wrote the correct value. This kind of corruption can remain hidden until an export, integration, or migration consumes the field differently. It was the second of two faults stacked in the same scenario: the first concealed it, and the next rerun exposed it as soon as the first was repaired. Diagnosis, code repair, and rerun proceeded without human intervention. The deterministic driver—not the agent that wrote the fix—issued the verdict.

What it does not do. EDD is a bounded write-path gate, not a universal verifier.

• A pass applies only to the identified build, evaluated runtime profile, oracle version, current Prove-lane evidence, and versioned coverage model.
• It detects only behavior that was modeled and exercised. Choosing what to cover remains human work.
• Matching the gold state shows that the application reproduced the reference; it does not show that the reference reflects business intent.
• Full-corpus validation cannot compensate for an interface path that was never exercised.
• The gate does not assess security, privacy, permissions, performance, resilience, or external outcomes such as whether an email was delivered.
• It does not replace code review or judgment about architecture and risk.

EDD replaces one narrower burden: manually reconstructing from a code diff what every stored value should have been. Responsibility for coverage, reference data, architecture, and risk remains outside the gate.

The conclusion

EDD sits above any individual test suite. Tests contribute evidence inside the box; the protected expectation and deterministic gate govern acceptance. EDD helps us trust agent-written code without pretending that we understand every internal step. The implementation is the X in the middle. The more tightly we define the box around a write path—its inputs, expected persisted state, runtime profile, and coverage—the more confidence we can place in its behavior when actual equals expected, and the less we need to care how X achieves it.

That confidence extends only as far as the box we have drawn. Human judgment still determines the expectations, the boundaries, and whether they are sufficient for the risk. Within those bounds, EDD makes trust answerable to evidence.