Fake court decisions from AI almost made it into judicial order.
A federal judge in California has sanctioned two law firms for submitting a legal brief containing fake citations generated by AI tools. Judge Michael Wilner described the AI-generated references as ‘bogus’ and fined the firms $31,000, criticising them for failing to properly check the sources.
The legal document in question was based on an outline created with Google Gemini and AI tools within Westlaw.
However, this draft was handed off to another firm, K&L Gates, which included the fabricated citations without verifying their authenticity.Judge Wilner noted that at least two cases referenced in the filing did not exist at all.
He warned that undisclosed reliance on AI could mislead US courts and compromise legal integrity. This case adds to a growing list of incidents where lawyers misused AI, mistakenly treating chatbots as legitimate research tools.
The judge called the actions professionally reckless and said no competent attorney should outsource research to AI without careful oversight.
Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!