This week, a lawyer in Western Australia was ordered to pay more than $8,300 after they were found to have used AI to prepare court documents.
The documents contained citations for cases that did not exist.
It follows a Victorian Supreme Court murder case in which lawyers submitted misleading, AI-generated material to the court.
Since 2023, there have been at least 20 cases of false, AI-generated submissions in Australia’s courts.
Here’s how the use of generative AI is impacting Australia’s court system.
Context
According to a 2024 report from legal research company Thomson Reuters, nearly a third of Australian legal professionals use generative AI tools to assist with tasks like drafting emails, letters, and contracts.
A recurring problem is that language models can produce cases, judgments, or quotes that seem real, but are entirely fake.
French researcher Damien Charlotin has recorded 307 incidents of fake, AI-generated content being submitted in court since May 2023, most of them in the U.S.
UNSW legal expert Professor Michael Legg told TDA that problems emerge when people use AI models for research without verifying results.
“I think we’re still going to see more cases coming out,” he said.
Professional bodies have warned against over-dependance on AI.
The NSW Bar Association advises lawyers to “verify the accuracy, reliability, and currency of AI-generated information... before relying on it”.
Recent cases
Victoria:
Defence lawyers apologised to the Supreme Court last week, after they filed documents that included fake quotes and non-existent case judgments generated by AI.
The documents were initially submitted in the trial of a teen boy who was found not guilty of the 2023 murder of a 41-year-old woman. Once the errors were discovered, the documents were re-filed.
Presiding Justice James Elliott said: “It is not acceptable for AI to be used unless the product of that use is independently and thoroughly verified.”
Your contribution ensures The Daily Aus can continue doing the work you love.
I want to give $10 monthly to help TDA keep doing the news.
Your details:
WA:
A lawyer was referred to the state’s Legal Practice Board, and ordered to pay costs, after his submissions in an immigration case were found to include four citations to cases that did not exist.
The lawyer said he used Claude AI as a research tool, and then “validated” the submissions using Microsoft’s Copilot.
“I had an incorrect assumption that content generated by AI... would be inherently reliable,” he told the court.
Justice Arran Gerrard said the incident “demonstrates the inherent dangers [of] solely relying on the use of artificial intelligence in the preparation of court documents.”
Court guidelines
Each state’s courts set their own rules on AI use. So far, these include:
- NSW: The Supreme Court bans the use of AI to generate certain evidence documents.
- Vic: The use of AI to prepare documents should be disclosed.
- Qld: People representing themselves are warned that AI is not a substitute for legal advice.
- WA, SA, ACT and the Federal Court: Formal guidelines are being developed.
Non-lawyers
According to Charlotin’s database, more than two-thirds of the Australian cases involving fake AI-generated content were from people representing themselves in court.
A Thomson Reuters report found the “unauthorised practice of law” (where non-lawyers use AI in place of real legal representation) is widely seen as the biggest risk AI poses to the legal profession.
Industry impact
The tasks completed by generative AI in a legal setting are often undertaken by paralegals and junior lawyers.
Data provided to TDA by job search platform GradConnection showed the number of law graduate roles advertised is down 6.1% from pre-COVID levels.
Legg said he doesn’t expect AI to “completely replace people,” but there may be less of a need for junior roles.
However, he said humans are still needed to check AI and to interact with clients, which could mean future law grads need different skill-sets.
