The lesson that keeps you out of the news.
Every sanctioned lawyer in every AI citation case had one thing in common: the output looked right. This is a repeatable method for checking legal AI work, built around the specific ways it fails.
01 How this actually goes wrong
The famous failure mode — a chatbot inventing a case that never existed — is the easy one. It is embarrassing, it is well publicised, and modern legal platforms with grounded citations have made it much rarer.
The failure that survives is subtler and more dangerous: a real case, correctly cited, that does not say what the summary says it says. Or says it in a different jurisdiction. Or said it before it was overturned. The link works. The case exists. You click it, see a real opinion, and your brain records the citation as verified without reading the holding.
Grounded citation makes output more trustworthy and harder to check, at the same time. A visible source link satisfies the part of your brain that was doing the doubting. That is precisely when a verification habit has to be procedural rather than instinctive.
02 Where models fail predictably
These are not random errors. They cluster, and knowing the clusters tells you where to look.
| Failure | What it looks like |
|---|---|
| Confident interpolation | A gap in the source material gets filled with the most plausible-sounding content. Reads seamlessly. This is the big one. |
| Negation drift | "The court declined to extend" becomes "the court extended". Single words that invert meaning are disproportionately likely to flip. |
| Jurisdictional blending | Authority from an adjacent jurisdiction presented as though local, because the reasoning is similar and the model is optimising for a good answer, not a correct forum. |
| Currency failure | Correct as of some point in the past. Superseded statutes and overruled decisions read exactly like current ones. |
| Convenient framing | You asked a question that implied a desired answer, and you got it. Models are agreeable. This is the failure your own bias hides best. |
| Quantitative slippage | Numbers, dates and defined terms drift between a source document and a summary of it, especially across long documents. |
Two AI-drafted memos land on your desk. Memo A cites eleven authorities, all with working links. Memo B cites three and hedges twice about uncertainty. Which needs more of your verification time?
Answer before opening.
Memo A — and it is not a close call. Eleven working links create eleven opportunities for a real case to be mischaracterised, and the volume itself suppresses scrutiny; nobody opens all eleven. Density of citation reads as rigour and functions as camouflage.
Memo B has told you where it is unsure, which is the single most useful thing a draft can do. Hedged output is easier to supervise than confident output, which is an argument for prompting in ways that permit uncertainty rather than demanding a clean answer.
03 A verification order that works
Verification fails when it is a vague intention to "check things". Make it a sequence, run in this order, because the order is doing work — it front-loads the checks most likely to catch a career-relevant error.
The pass, in order
- Open every authority. Not the link preview — the document. Read enough of the holding to confirm it supports the proposition, not merely that it exists.
- Check currency. Still good law? Still the operative version of the statute? This is a mechanical step and it is skipped constantly.
- Confirm jurisdiction and posture for each authority. Right forum, and the procedural posture actually comparable.
- Trace every number and date back to a source document. Do not accept a figure that appears only in the summary.
- Hunt negations. Search the draft for not, except, unless, declined, failed to. Verify each against the source individually.
- Read for convenience. Ask where the draft is suspiciously helpful to your position, and check those passages hardest.
- Name the gap. What did it not address? Absence is invisible on a read-through and is where interpolation hides.
04 Prompting so the checking is easier
Verification burden is set at the prompt, before a word is drafted. Three habits cut it substantially.
Ask for uncertainty explicitly. Tell it to flag anything it is not confident about and to state where the source material is silent. You are trading a small amount of polish for a map of where to look.
Ask for the counter-argument. Requesting the strongest case against your position surfaces authority that a one-sided draft omits, and omission is the failure mode that a read-through cannot catch.
Separate retrieval from argument. First ask what the sources say. Check that. Then ask for the argument built on it. Combining both in one request is what allows a plausible-sounding claim to arrive pre-wrapped in supporting citations.
05 Who is responsible
Worth stating plainly, because it is the whole point. The professional duties — competence, confidentiality, supervision, candour to the tribunal — attach to the lawyer. They do not transfer to a vendor, a model, or a colleague who ran the prompt. No terms of service move them.
Which means verification is not a productivity tax on AI use. It is the professional work. The machine took the drafting; what remains is judgement, and judgement is the part that was always billable.
This week's challenge
Take one AI-assisted document you have already reviewed and considered finished. Run the seven-step pass on it properly. Most people find at least one thing — usually a date, a number, or a citation that supports a narrower proposition than claimed. Finding it on your own desk is considerably better than the alternative.