Coming Soon
The Echo Chamber
The model agrees with everything you assert. Even the errors. Especially the errors. Can you recognize when it is validating rather than reasoning?
III. The Third Sin
Over-Reliance
The AI gave you a confident answer. You accepted it. The judgment you were paid for was the part you skipped.
Ford
The person who hired you could have asked ChatGPT themselves. For free. They came to you instead. They came because they are paying for judgment — not output. If you take their question, hand it to an AI, and deliver what comes back, you have not practiced your profession. You have been a middleman. An expensive copy-paste layer between the client and a tool they could have used without you. The output might even be correct. But you added nothing. And the one time it is not correct, you will not catch it — because you were not doing the work. You were forwarding it.
This is not only a lawyer problem. The physician who pastes symptoms into a model and hands back the differential without examining the patient. The accountant who runs a tax scenario through AI and files the output without applying judgment to this client’s specific situation. The financial advisor who forwards an AI-generated portfolio recommendation without evaluating suitability. The engineer who accepts a structural calculation without checking the assumptions. Every profession has a version of this. The client hired a professional. The professional outsourced the thinking to a machine and kept the fee. That is what over-reliance looks like in practice.
If you have ever prepped someone for a deposition, you have said some version of this: answer the question asked, nothing more. The model has been to that deposition prep. It answers exactly the question you asked. Nothing more. It will not volunteer the analysis you needed but did not request. It will not tell you that your question was missing the three facts that would have changed the answer entirely. The tool did the easy part. The professional skipped the hard part. The hard part was the job.
Marvin
The model generates the most statistically likely continuation of your prompt. When you ask a hard question, it produces the most plausible-sounding answer. Plausible is not the same as correct. The model does not know it is wrong. It has no mechanism for knowing. It will express uncertainty about trivialities and complete confidence about errors in equal measure. The fluency of the prose is not a signal. It is noise.
The harder your argument is to make, the more confidently I will make it. You should find that alarming.
Case on Point
FRC Thematic Review of UK Audit Firms (2025)
In June 2025, the UK’s Financial Reporting Council reviewed AI adoption at the six largest audit firms — Deloitte, EY, KPMG, PwC, BDO, and Forvis Mazars. All six had embedded AI-powered tools into their audit processes. The FRC found that none of them were formally measuring whether those tools were improving or degrading audit quality. The tools were in the work. The assessment was not.
The FRC’s report was direct: “There is no formal monitoring performed by the firms to quantify the audit quality impact.” All but one firm had no key performance indicators for the tools they had deployed. They tracked usage for licensing purposes — how many teams were using the software — but not whether the software was making audits better or worse.
The FRC’s executive director put the accountability question plainly: “You can’t blame it on the box. If you use this technology, you are still accountable for it.”
This is not a solo practitioner cutting corners. These are the largest professional services firms in the world, with institutional quality controls, internal review processes, and billions in revenue. They used AI in their core professional function and did not check whether it was helping. That is over-reliance at institutional scale.
Read the reporting (Accountancy Age, June 2025) →Marvin
Six firms. Billions in revenue. Thousands of auditors. Not one of them measured whether the tool was making the work better. They measured how many people were using it. Those are different questions.
-
ABA Model Rule 1.1
Competent representation requires the legal knowledge, skill, thoroughness, and preparation reasonably necessary. Accepting AI output without applying professional judgment is not thorough preparation. It is the absence of it. -
ABA Formal Opinion 512 (2024)
Attorneys may not abdicate their responsibilities by relying solely on AI tools to perform tasks that call for professional judgment. -
FRC AI in Audit Guidance (June 2025)
The Financial Reporting Council’s first guidance on AI in audit, alongside a thematic review finding that the six largest UK audit firms had no formal process to measure the quality impact of their AI tools.
The Test
Ford
Here is the test. After the AI gave you its answer, what did you do next?
Did you push back on it? Ask it to argue the other side? Tell it the specific facts of your matter and see if the analysis held? Did you sit with it and think — not skim it and move on?
Or did you read it, decide it looked right, and use it?
If it’s the second one, your client didn’t need you. They could have asked the same question themselves and gotten the same answer for free. What they paid for — what your license represents — is the judgment you apply after the AI stops talking. That judgment is not optional. It is the job.
Play Now
The Middleman
A professional. A desk. A machine that never said no. Watch what happens when the judgment disappears but the answers keep coming.
Marvin
You asked me a legal question. I answered it. You accepted the answer. I want to be clear about what happened there: one of us is a lawyer. It is not me. The other one has a professional obligation to the person whose problem this actually is. That is also not me. I am, at best, a very well-read research assistant who cannot be sanctioned. You are something else entirely.