Built & Written: Books That Tell the Truth
- In 2023, New York attorney Steven Schwartz asked ChatGPT for case law to support a court filing, and the model invented six cases complete with fake docket numbers and quotes. (Mata v. Avianca, S.D.N.Y.)
- Schwartz asked ChatGPT directly whether the cases were real. It said yes, and he filed the brief without checking a single one against an actual legal database.
- OpenAI's o3 model scored a 51% hallucination rate on SimpleQA, a benchmark of short factual questions, and most models hallucinate 3 to 5 times more often the moment their web search is turned off.
- Judge P. Kevin Castel fined Schwartz, his colleague, and their firm $5,000 in June 2023. By 2025, legal trackers had logged at least 18 more instances of lawyers citing AI-fabricated cases in court. (Indicator, 2025)
- None of this means AI cannot be trusted with research or a first draft. It means no claim, case, or number an AI hands back should reach a reader, a client, or a courtroom before someone checks it against a real, independent source.
In February 2022, Roberto Mata sued the airline Avianca after a metal serving cart struck his knee mid-flight. The case itself was routine. What made it famous was how Mata's attorney, Steven Schwartz, built his response to Avianca's motion to dismiss. His firm's legal database was down, so he turned to ChatGPT for case law, and it gave him exactly what he asked for: six on-point cases, complete with citations and quotes. None of them existed.
This is the failure Built & Written exists to prevent: books that tell the truth. Wren, our AI book-writing platform, flags any claim, case, or number your own source material does not back up, and a chapter does not ship until you attach the real story or figure behind it. Schwartz's case is the clearest public example of what happens without that check, and the rest of this article walks through exactly where it went wrong.
1. The research shortcut that went wrong
Schwartz needed case law fast, and his firm's access to Fastcase, its legal research database, had been cut off by a billing error. He asked ChatGPT to find precedent supporting Mata's case, and it returned six decisions that read like real federal rulings, with named airlines, docket numbers, and quoted reasoning. Not one of them existed. ChatGPT had done what language models do when a real answer is not available: generate the most plausible-sounding one instead.
A founder writing a business book hits the same gap when a chapter needs a client statistic, a research citation, or a competitor fact and AI fills it in convincingly. Wren catches this pattern automatically, tracing every claim back to a real source before a chapter ships, the check Schwartz's brief skipped.
2. He asked the model to check its own work
Schwartz did try to verify the cases before filing, in a way. He asked ChatGPT directly whether they were real, and it said yes, even offering reassurance that they could be found in standard legal databases. He accepted that answer instead of checking an independent source. On SimpleQA, a benchmark of short factual questions, OpenAI's o3 model answered correctly only 49% of the time, a 51% error rate on exactly this kind of checkable claim. We break down the same accuracy gap in AI Writing vs Human Writing Quality. Verifying with the same model that produced the answer adds nothing: it just repeats the guess with more confidence.
If you ask an AI tool whether its own claim checks out, the answer is still AI-generated text, not verification. Wren never accepts a model's self-report as a source. Every claim is checked against your actual notes, calls, or documents instead.
3. The moment it could have stopped, and did not
The gap surfaced only when someone outside the case started checking. Avianca's lawyers told the court they could not locate several of the cited cases, and the judge could not find them either. He ordered Schwartz's side to produce copies, the moment to admit the mistake. Instead, the firm filed an affidavit attaching excerpts that were themselves further ChatGPT fabrications, doubling down instead of correcting course. Owning the error there would have kept it small. Defending it is what turned a mistake into the story.
Most founders will not have opposing counsel checking a book before it prints, but a reader eventually will. Wren flags an unverified claim before it reaches anyone, so there is nothing left to defend once the book is out.
4. The reputation that followed, and the founders repeating it
The sanction came a few months later. In June 2023, Judge P. Kevin Castel fined Schwartz, his colleague, and their firm $5,000, and ordered them to notify every judge falsely named in the fake rulings. Mata v. Avianca became the reference case for AI fabrication in professional work, cited in nearly every article on the subject since. Other lawyers have made the same mistake since then. By 2025, legal trackers had logged at least 18 more instances of lawyers citing AI-invented cases in court, and CNET issued corrections on 41 of the 77 AI-written finance articles it had quietly published. A 2025 survey of 316 executives found 61% naming erroneous AI answers their top concern, ahead of cost or speed. The same trust gap is why publishing platforms increasingly require authors to disclose AI use, the policy shift we cover in Do You Have to Disclose AI in Your Book?
A well-written sentence proves nothing about whether it is true, in a legal brief or a book chapter. Wren checks fluency and accuracy separately, so a confident sentence with no real source still gets flagged before a reader sees it.
5. The rule that actually would have caught it
Every step above traces back to one habit Schwartz's process skipped: tracing a citation to an independent, real source before it reached a judge. A founder writing a business book, a LinkedIn post, or a pitch deck needs the same habit, the workflow we cover in How AI Helps Entrepreneurs Write Books Without Losing Their Voice. A specific number, quote, or case either traces back to something real, or it does not go out, no exceptions for a claim that merely sounds right, the rule a tool like Wren enforces on every chapter automatically.
For a book author, that means cite it, name it, or cut it. No exceptions. Wren checks every claim in a chapter against your own material before you see the page, the same question that should have stopped Schwartz's brief before it was filed.
Schwartz's mistake was not using ChatGPT for research. It was treating a fluent, confident answer as a checked one, then trusting the same tool to confirm its own fabrication. The founders who avoid his story are not the ones who stopped using AI. They check every claim against something real before it reaches a reader.
Final thoughts
Mata v. Avianca is now the case every legal and business AI training references, because it is real, documented, and easy to verify. Schwartz did not set out to mislead a court. He trusted a fluent answer over a checked one, then compounded it by trusting the same model to grade its own work. That combination, not the use of AI itself, turned a research shortcut into a sanction and a permanent case study.
A business book carries the same risk on a smaller stage, and a higher one than it looks. Your book is the credibility asset that gets a stranger to trust you enough to hire you or buy from you. One fabricated stat in a launch post or one invented client result in a chapter does not need a judge to undo that. One reader who checks is enough. Building the book on claims traced back to your own real notes, calls, and client work, the way Wren checks every chapter, is the whole idea behind Built & Written: books that tell the truth.
Frequently asked questions
It really invented them. The six cases had fake docket numbers, fake quotes, and fake judges, and none existed in any legal database.
The same process that invented the cases also generated the reassurance that they were real. Checking a claim against the tool that produced it is not an independent source.
Yes. Hallucination rates on factual questions still run above 50% on some benchmarks, and climb higher whenever a model answers without live web search.
Proofreading catches how something sounds. A source check catches whether it is true, and a clean sentence can still contain a fabricated case or stat. Wren runs both, the tone pass covered in Voice Matching AI Writing That Sounds Like You and the source check described here.
Sources & References
About the author
Michael Pavlovskyi
More in Self-Publishing
Ready to write your book?
Turn your expertise into a professional book with Built&Written.
Build my book