News Feed

Barely a week goes by without an artificial intelligence (AI) headline of some kind. Generally, when we speak about AI, we speak about so-called “Large Language Models” (or LLMs) such as OpenAI’s Chat GPT or the recently launched Chinese competitor “DeepSeek”. Modern LLMs are remarkable in their ability to present as both human and expert, providing near-instant answers to complex questions. AI solutions are already widely used in a legal context.

However, as with any emerging technology, it is vitally important to be aware of the limitations of AI and LLMs. I have previously spoken (in a video on LinkedIn available here) on some of the risks which arise in the context of AI assisted litigation. One such risk is that LLMs are susceptible to “hallucination”. Put simply, when asked a legal question there is a risk that an LLM will spit out an authority (such as a reported case, complete with apparently credible citation) that does not exist.

This is deeply problematic, particularly in jurisdictions (like England) where our system of law is developed and created through case law. If a party submits: “Smith v Jones [1983] AC 23 supports my position”, that party’s assertion may become law, notwithstanding whether Smith v Jones is a real case, or not (…it’s not).

The recent (and real) case of Olsen v Finansiel Stabilitet A/S demonstrates how seriously the Court takes the issue of fake cases being used in submissions. The substance of the case relates to the application of foreign law limitation periods when seeking to enforce a judgment in England. Finansiel Stabilitet A/S, a Danish financial institution sought to enforce a Danish judgment against Mr and Mrs Olsen, worth about €5.8m and 1.25m Danish kroner. For reasons I will not discuss here, Mr. and Mrs. Olsen were successful on appeal in demonstrating that the judgment had expired under Danish law and therefore should not be enforced in England.

However, in doing so, Mr. and Mrs. Olsen (acting in person but with lawyers providing some assistance in the background) relied on the case of Flynn v Breitenbach [2020] EWCA Civ 1336, and provided a two page summary of the case in their authorities bundle. It became apparent that Flynn v Breitenbach did not exist. It is not clear whether Flynn v Breitenbach was hallucinated by some LLM; Mr. and Mrs. Olsen informed the Court that the case and summary had been sourced from a “European business network” formed of law firms and three major banks, and that the summary had been sourced from “their internal databases”. 

Mr. and Mrs. Olsen apologised to the Court. They explained that they had been poorly advised and (after all) were representing themselves without lawyers “on the record”. They were 63 and 67 years old, “unemployed and living at the flat in Guildford, on universal credit. They have applied for jobs, without success. He has had medical issues, with long covid.” Mr Justice Kerr also found that “they have in other respects behaved properly during this litigation, observing the usual courtesies and cooperating reasonably well with the respondent's solicitors. They have, fortunately for them, gained no advantage from the case summary because its inauthenticity was patent.”

With all that mitigation, Mr Justice Kerr “narrowly and somewhat reluctantly” determined that Mr and Mrs Olsen should not be summonsed for contempt of court (i.e. subject to criminal prosecution for their actions with the possibility of fine or imprisonment if found to be in contempt). Instead, the Court ordered that (despite them plainly being the successful party) they should not be entitled to recover any of their costs of the appeal.

Whilst we cannot be sure whether an LLM was responsible for creating Flynn v Breitenbach, LLMs have hallucinated cases which have been presented to Courts (and widely reported) in other jurisdictions. In another case, things could have been far worse for Mr and Mrs Olsen, and they may have found themselves subject to criminal proceedings. This case stands as a timely reminder of the limitations of LLMs when it comes to litigation, and the serious risks of getting it wrong.