Substantive testing in the AI era
Substantive testing is the part of an audit that goes to the records themselves: verifying amounts, balances, and transactions directly against the underlying evidence, rather than relying only on controls. It is how an auditor earns the signature — professional skepticism exercised against the actual data. The question the AI era adds is simple to state and hard to engineer: how do you accelerate that work with a model without breaking the property that makes it defensible — that the auditor verified it?
Last updated July 24, 2026
The defensibility property
No auditor can sign an opinion on something they can't verify. Any AI workflow that answers “trust me” — a model reads the evidence and asserts a conclusion — breaks the property at the core of the profession. The test of an AI-era substantive workflow is not how much it automates but whether the auditor can still check every finding against the records and stand behind it personally.
The defensible design
Three architectural choices preserve the property. First, the model drafts, never concludes: the auditor asks in plain language, the AI writes the query. Second, every query — drafted or hand-written — passes a deterministic validation gate before it touches evidence: locked to read-only, checked against the engagement's actual schema, and trial-run, so anything that wouldn't execute cleanly is caught and explained first. Third, identifying values are tokenized before any model interaction — the model reasons over consistent stand-ins, and the real values render only on the auditor's screen.
Where it fits
Substantive testing pairs with full-population control testing — the controls run measures every in-scope record, and the substantive workspace is where the auditor proves individual findings against the underlying rows. Both stand on the same rule: the machine measures; the auditor verifies and owns the opinion. The experience of that work — where it binds and what it costs — is part of what the AuditEx Program measures.
- Substantive testing verifies records directly — it is the auditor's own check.
- Defensible AI design: the model drafts; it never concludes.
- Every query passes deterministic validation — read-only, schema-checked, trial-run.
- Identifying values are tokenized before any model interaction.
FAQ
- What is substantive testing?
- Audit procedures that verify amounts, balances, and records directly against the underlying evidence — going to the records themselves rather than relying only on controls. It is how an auditor earns the right to sign: they checked it.
- Can AI be used in substantive testing without breaking defensibility?
- Yes — if the architecture keeps the auditor as the verifier. A defensible design has the AI draft queries the auditor asked for in plain language, passes every query through a deterministic validation gate (read-only, schema-checked, trial-run) before it touches evidence, and tokenizes identifying values so no real identifier ever reaches a model. The auditor still verifies the records and owns the conclusion.
- What should a reviewer ask about an AI-assisted substantive workflow?
- Three questions: Can the auditor see and verify every record behind every finding? Is every generated query validated deterministically before execution? Do identifying values ever reach the model? A workflow that answers yes, yes, no is built for defensibility.