Replaying Against the Framework in Force: Inspection-Time Fidelity

1. The Clause That Does All the Work

Across this series the test for a replayable decision is stated the same way: can you reconstruct the path a decision walked, under the rules in force at the time? The last clause is easy to nod past and is the entire difficulty. It is not enough to replay that a decision happened, or even how it happened mechanically. The replay has to resolve against the ontology and policy version that governed the decision when it was made — not the version current when the regulator finally asks.

We call this property framework-fidelity, and replaying without it produces something worse than no replay: a confident, plausible reconstruction that is quietly wrong, because it judged a past decision by present rules.

2. Ontology Drift: Why the Ground Moves

The definitions a system reasons with do not stand still. Risk categories get redrawn, thresholds get retuned, policies get amended, the meaning of a label shifts. This gradual movement — ontology drift — is normal and usually invisible. It becomes a problem only at the moment of inspection.

Consider a decision made in early 2026 under one definition of a risk band, inspected in late 2027 after that band has been redefined twice. An audit trail that records only the inputs and outputs, or that replays against the current ontology, will map those same inputs to a category that did not exist when the decision was made. The reconstruction looks authoritative and misrepresents history. The only defence is to have pinned the decision, at the time, to the exact framework version that governed it.

3. How Framework-Fidelity Is Achieved

Framework-fidelity is an architectural property, not a promise. Three things have to be true:

• The framework — the policy and ontology the system reasons with — is versioned and content-addressed, so each version has a stable, verifiable identity.
• Each decision record is bound to the framework version in force at the moment it was made, so the record carries which rules applied.
• Replay resolves to that version, reconstructing the decision against the historical framework rather than the current one.

With those in place, a replay is faithful: it shows the decision as it was, judged by the rules it was judged by. This is the inspection-time companion to the offline verifier — the verifier proves the record is intact, and framework-fidelity proves the replay reflects the conditions of the time.

4. The Portability-versus-Fidelity Tension

Fidelity inside one system is hard enough. Making a fidelity claim that travels — so a regulator can check it independent of the vendor — raises a tension named publicly by Peter Borner, Chairman of the Open Proof Standards Foundation:

The fidelity-specific form of the tension is this: a portable standard wants to mandate enough decision-condition semantics that a fidelity claim can be expressed and checked anywhere. But the more semantics it prescribes, the closer vendor-neutrality drifts toward vendor-prescription. How much decision-condition semantics can a portable-proof standard mandate before portability becomes prescription? That is the open question, and it is the standards-layer's to resolve — Peter Borner's lane, not the substrate's.

5. The Question We Have Committed to Address

This is not a closed argument, and it should not be settled by one vendor. Quantamix Solutions has committed to a written submission in the OPSF public-comment process addressing two related tensions: portability versus fidelity, and inspection-time framework-fidelity replay against ontology drift. The division of labour is the same one this whole series rests on. The substrate builds the decision record that makes fidelity possible — versioned framework, bound to each decision, replayable. The standards layer, which is the OPSF's lane, defines the portable format that lets a fidelity claim travel and be checked by a regulator independent of either party.

The two compose, and neither claims the other's job — the discipline developed in proof precedes permission and recall vs verifiability. Framework-fidelity is what makes a replay honest; portable standards are what let that honesty be checked from the outside.

6. Where the EU AI Act Sits

Article 12 obliges record-keeping sufficient for traceability and to support the post-market monitoring of Article 72. On the text, it does not specify that replay must resolve against the framework version in force at decision time. That requirement is not written on the page — but it is what an honest reconstruction actually requires, and it is what an adversarial 2027 inspection will expose the absence of. Framework-fidelity sits above the legal floor: aligned with where the Act is going, not a box it tells you to tick, and never a claim of compliance or certification.

Continue Reading

• Recall vs Verifiability in EU AI Act Audit Trails
• The Offline Proof Verifier for AI Decisions
• Proof Precedes Permission: The Substrate Seam
• Methodology, Architecture & Standards in AI Governance

Frequently Asked Questions