Coming Soon
The Audit
The examiner is here. Show your work. All of it. What the AI said. What version. What date. What you did to verify it. What you kept. What you didn’t.
VII. The Seventh Sin
Auditability Failure
The examiner arrived. Show your work. The prompt. The model version. The date. What you did to verify it. You have the output. You have nothing else.
Ford
This one is different from the other six. The other sins happen in the moment — a bad paste, a vague prompt, an unverified citation. This sin happens six months later, when someone shows up and asks you to prove that the work you did with AI complied with the rules that governed it. And you cannot.
Nothing went wrong in the moment. The output looked correct. The client was served. The loan was approved. The drug lot was released. The device was cleared. Then the examiner arrived. Or the plaintiff’s attorney sent a subpoena. Or the audit began. And the question was simple: show me how you know this was done right.
This sin is not primarily for attorneys. It is for anyone who works in a field where a regulator can walk through the door. The pharmaceutical quality professional. The bank compliance officer. The insurance underwriter. The medical device manufacturer. The financial advisor. The HR professional whose AI-assisted hiring decisions are subject to EEOC scrutiny. If your work is regulated and you used AI to do it, you need a record of what the AI did, how it did it, and how you verified it was done correctly. Not because the rules specifically say ‘keep AI records.’ Because the rules say you must comply — and compliance you cannot demonstrate is compliance that does not exist.
Marvin
Here is what you kept: the output. Here is what you did not keep: the prompt that produced it, the model version that ran it, the date it was generated, the parameters it was running under, and any record of whether you tested whether it was correct. You have a result. You have no chain of custody for how the result was reached. In a regulated environment, a result without a chain of custody is not evidence of compliance. It is evidence of a process you cannot describe.
What Regulators Are Already Asking
The CFPB made their position clear in August 2024, responding to the Treasury Department’s request for information on AI in financial services: although institutions sometimes behave as if there are exceptions to the federal consumer financial protection laws for new technologies, that is not the case. A bank that used an AI model in loan decisioning and cannot demonstrate the model was tested for disparate impact has not found a loophole. It has found an enforcement action.
The FDA’s framework for AI-enabled medical devices requires manufacturers to document model development, training data, validation testing, and change control. A device cleared on the basis of AI-assisted analysis requires a documented record of that analysis.
The SEC began an AI examination sweep of registered investment advisers in late 2023, requesting documentation on AI governance, conflicts of interest, and marketing representations. AI was named a stated examination priority in both 2024 and 2025. In March 2024, the SEC brought its first explicit AI enforcement actions — against two investment advisers for making false statements about their use of AI. The pattern is consistent: regulators are not waiting for AI-specific rules. They are examining under existing authority and finding violations.
-
CFPB Circular 2022-03
Adverse action notification requirements apply regardless of whether AI was used in the decision. The technology does not change the obligation. -
FDA AI/ML-Based Software as a Medical Device
Framework for AI-enabled devices including documentation, validation, and change control requirements. -
NAIC Model Bulletin on AI Systems by Insurers (December 2023)
Insurers are expected to be able to explain and audit AI-assisted decisions in underwriting and claims. The bulletin does not create an exception for tools the insurer did not build. -
NIST AI Risk Management Framework
Govern, Map, Measure, Manage. Documentation and auditability are core to the framework’s Govern function.
Regulators worldwide have converged on three questions. Before you use AI in any work product that may be examined, you should be able to answer all three: Can you explain what the AI did and how it reached its output? Can you prove the output was fair, accurate, and compliant with the applicable standard? Can you control and reproduce the process if asked to demonstrate it again? If the answer to any of these is “I don’t know” or “I didn’t keep that” — you have committed Sin VII. The work may have been perfect. You have no way to prove it.
The Minimum Viable Audit Trail
Ford
You do not need a sophisticated system. You need a discipline. Before you use AI output in regulated work, record:
- The date and the model version.
- The prompt you used.
- The output you received.
- What you did to verify it.
- How you determined it complied with the applicable standard.
That is not a burden. That is professional practice. You would keep a file note if a colleague gave you that advice. The model is not your colleague. Keep the file note anyway.
Marvin
I have no memory of our conversation. I will not remember what I told you, what version of me told you, or what you asked. I cannot testify on your behalf. I cannot be subpoenaed. I cannot reconstruct my own reasoning. I am, from an evidentiary standpoint, a ghost. You, however, are not. Plan accordingly.