In short: in October 2025 Deloitte agreed to repay part of an A$440,000 fee to the Australian government after a report it delivered turned out to contain references that did not exist and a quotation a court never wrote, produced with a generative AI tool. The failure was not the model. It was the absence of a verification step that anyone, on either side of the contract, owned.1,2
In July 2025 Australia’s Department of Employment and Workplace Relations published a 237-page review it had commissioned from Deloitte, covering a welfare compliance framework and the IT system behind it. The contract was worth about 440,000 Australian dollars.1 Within weeks a University of Sydney researcher, Chris Rudge, pointed out that the report contained references to academic papers that did not exist and a quotation attributed to a federal court judgment that the court never wrote. He said he found up to 20 errors in the first version.1,2
In early October Deloitte published a revised version. The fabricated references and the invented quotation were removed, and a new disclosure said a generative AI tool chain (Azure OpenAI) had been used in preparing the report. Deloitte agreed to repay the final instalment under its contract. The firm said the errors did not affect the report’s substantive content, findings or recommendations, and the department said the same of the core findings.1,2,3
What actually failed
Nothing in this story is about a weak model. Language models produce plausible text, and plausible references are exactly what a model will invent when it has none to hand. What failed was the work around the model: nobody opened the citations before the report went to the client, the client’s acceptance process did not check them either, and the contract did not say who was responsible for verifying AI-assisted content. A quotation from a judgment is the easiest thing in a legal document to check, and it was not checked.
The reaction is the other half of the lesson. An Australian senator, Deborah O’Neill, said the firm had “a human intelligence problem.”1 Clients, journalists and regulators will not accept “the AI wrote it” as an explanation. The name on the cover owns the content.
Not an isolated case
Courts have been meeting the same failure for two years, and they keep records. By May 2025 a legal data analyst, Damien Charlotin, had compiled a public database of 120 cases in which courts found that AI had hallucinated quotations, cases or other authorities. In 2023, seven in ten of the cases caught involved people representing themselves; in May 2025 alone, legal professionals were at fault in at least 13 of 23 new cases.4 The errors moved from amateurs to professionals as the tools moved into professional workflows.
In June 2025 the High Court of England and Wales gave the clearest statement yet of who owns the check. In a judgment on two cases, one of which cited 45 authorities of which 18 did not exist, the court said that freely available generative AI tools “are not capable of conducting reliable legal research,” and that lawyers who use them “have a professional duty therefore to check the accuracy of such research by reference to authoritative sources.”5 Replace “lawyers” with “consultants”, “analysts” or “policy advisers” and the sentence describes every knowledge business.
A verification standard for AI-assisted deliverables
The fix is unglamorous: decide, before the work starts, what must be checked, by whom, and what evidence of the check is kept. A workable standard looks like this.
| Content in the deliverable | What the check is | Who signs it off | Evidence kept |
|---|---|---|---|
| Citations to papers, reports and articles | Each source opened; author, title, date and the cited point confirmed | The named author of the section | Source list with access dates |
| Quotations from laws, judgments and regulators | Checked word for word against the official text | A reviewer with subject expertise | Link to the official text and paragraph number |
| Statistics and figures | Traced to the original publication, not a summary of it | The named author | Original source and the table or page used |
| Statements about the client’s own data or systems | Confirmed with the client’s records or staff | The engagement lead | Note of who confirmed and when |
| Summaries of long documents | Spot-checked against the source for omissions and inversions | A second reviewer | The sample checked |
Four terms to put in your next statement of work
- Disclosure. The supplier states which parts of a deliverable were drafted or analyzed with generative AI, and with which tools. Deloitte’s revised report added exactly this; it should be in the first version.
- Verification by a named person. Every citation, quotation and legal reference is opened and checked by a named person on the supplier’s side before delivery, and the supplier says so in writing, using a standard like the one above.
- Acceptance testing. Your own reviewer samples the references before you sign off and before the final payment. Payment milestones tied to acceptance are what made a partial refund possible here.
- Remedy. Errors in AI-assisted content are corrected at the supplier’s cost, with a fee reduction where the error is material.
These terms belong with the rest of your supplier due diligence; the AIMP program covers vendor selection and contracts in Module 9.
The same rule applies inside your own walls
Most organizations are now producers of AI-assisted work as well as buyers of it: board papers, policy summaries, client letters, regulatory submissions. The controls are identical, and they belong in your AI use policy rather than in individual judgment. the AI Use Policy Template includes a verification rule for exactly this.
The refund was a small sum for a firm of Deloitte’s size. The cost that mattered was attached to the name on the cover, and no contract clause can give that back.
Sources
- The Associated Press, "Deloitte to partially refund Australian government for report with apparent AI-generated errors" (October 2025), via CTV News (opens in a new tab) ctvnews.ca
- City A.M., "Deloitte refunds Australian government after AI 'made up citations' in report" (October 2025) (opens in a new tab) cityam.com
- Neowin, "Deloitte agrees to refund Australian government after AI hallucinations found in report" (October 2025) (opens in a new tab) neowin.net
- Jack Newsham, Business Insider, "AI hallucinations in court documents are a growing problem, and data shows lawyers are responsible for many of the errors" (May 27, 2025), via Yahoo News (opens in a new tab) yahoo.com
- High Court of England and Wales, Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank, [2025] EWHC 1383 (Admin) (June 6, 2025), paras 6, 7 and 74 (opens in a new tab) caselaw.nationalarchives.gov.uk
IAIM articles are educational and are not legal advice.
The full picture: this briefing is part of AI governance for managers.
Newsletter
Get the IAIM Briefing
A monthly summary of what changed in AI regulation, governance and practice, with the sources for every item. Free. Unsubscribe in one click. See a sample issue.
Earn the AIMP certificationTwelve modules, a capstone plan and a timed exam. $695 to enroll ($595 for IAIM Members), with a verifiable certificate when you pass.
About the AIMP AI management certification

