TL;DR
Five instances in recent years highlight how AI hallucinations have led to official errors and embarrassment for governments. These incidents underscore the need for human oversight of AI-generated content in official materials.
Five cases over the past two years have confirmed that AI hallucinations—fabricated or incorrect outputs from AI systems—have caused embarrassment and errors for governments worldwide, highlighting risks associated with unverified AI use in official documents.
In April 2025, South Africa withdrew its Draft National Artificial Intelligence Policy after discovering that at least six of its 67 cited sources were AI hallucinations, including fictitious academic journals. South Africa’s Minister of Communications and Digital Technologies, Solly Malatsi, stated that the inclusion of AI-generated citations was due to a lack of proper verification, marking the first time a government has retracted a document over AI hallucinations.
Similarly, in May 2025, the Trump administration released a report on children’s health containing numerous incorrect citations, some with URLs marked by ‘oaicite,’ indicating AI involvement. White House spokesperson Karoline Leavitt dismissed these as formatting issues, promising a corrected version, which was later published.
In August 2025, the Australian Department of Employment and Workplace Relations commissioned a report from Deloitte, which was found to contain fake references and made-up quotes. Deloitte acknowledged that AI tools had produced inaccurate citations, and the firm refunded the government $290,000 after correcting the report in September.
Canada faced a similar issue when Deloitte produced a 526-page healthcare report for Newfoundland and Labrador, which included fabricated citations. The report was reissued after corrections, and the government amended its procurement policies to require full disclosure of AI use and risk assessments.
Europe’s cybersecurity agency ENISA admitted that two of its threat reports from 2025 contained numerous AI hallucinated sources—26 out of 492 footnotes in one report were false—raising concerns about the epistemic reliability of AI-generated content in official publications.
Why It Matters
These incidents demonstrate the tangible risks AI hallucinations pose to government credibility, accountability, and decision-making. They highlight the urgent need for rigorous human oversight and verification processes when integrating AI into official documentation, especially in sensitive fields like policy, health, and cybersecurity. The cases also raise questions about the reliability of AI tools in high-stakes environments and the potential for misinformation to spread through institutional channels.
AI Programming Made Practical: A Step-by-Step Guide to Building AI-Powered Applications, Writing Better Code Faster, and Using Modern AI Tools with Confidence
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Background
Over the past two years, AI-generated misinformation has increasingly infiltrated official government documents, often unnoticed until errors are publicly exposed. The incidents span multiple countries and sectors, reflecting a broader challenge of integrating AI responsibly. South Africa’s withdrawal of its policy document marked a historic first, but similar errors have occurred in the US, Australia, Canada, and within European cybersecurity agencies, revealing systemic issues in AI oversight and verification processes.
“There will be consequence management for those responsible for drafting and quality assurance.”
— Solly Malatsi, South Africa’s Minister of Communications and Digital Technologies
“ENISA let AI touch the one layer it should never touch unguarded: the truth layer.”
— Chiara Gallese, AI law and data ethics researcher
AI Workflow Tools for Researchers & Analysts: Automating Literature Reviews, Summaries, and Hypothesis Generation with ChatGPT, Claude, and Perplexity
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What Remains Unclear
It remains unclear how widespread the use of AI in official government document creation currently is, and what specific safeguards are being implemented to prevent future hallucinations. The full extent of the impact of these errors on public trust and policy outcomes is still being assessed, and ongoing investigations may reveal additional cases.
Hallucination-Aware AI for Truthful and Aligned Systems
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What’s Next
Governments are expected to tighten AI verification protocols and increase human oversight in document drafting. Future reports and policies will likely include explicit disclosures of AI use, and oversight bodies may develop standards for AI content validation to prevent recurrence of hallucinations.
The AI-Powered Paralegal: Practical Tools and Prompts for Legal Research, Contract Review and Document Drafting (: The Paralegal AI Trilogy)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
What are AI hallucinations?
AI hallucinations are false or fabricated outputs generated by AI systems, such as fictitious citations, quotes, or data that appear credible but are entirely invented.
Why do these errors matter for governments?
Such errors can undermine the credibility of official documents, mislead policymakers, and erode public trust in government institutions and their use of AI technology.
Are these incidents isolated or part of a larger trend?
While these are notable cases, experts warn they indicate a broader challenge of ensuring AI reliability in high-stakes settings, especially without strict verification protocols.
What is being done to prevent future AI hallucinations?
Governments and organizations are expected to implement stricter human oversight, verification steps, and disclosure requirements for AI-generated content in official documents.
