Ghost Citations

An advocate’s signature on a court filing is more than an administrative seal; it is a personal representation to the court that the document does not mislead, its citations are real, and every authority named above the signature exists and has been read.

The document is not merely submitted; it is vouched for. This is an irreducible obligation of the profession, and it is the precise point at which artificial intelligence (AI) presses.

In Kenya, the warning has already come from the bench. In May 2025, at the Judiciary’s Digital Transformation Conference, Justice Isaac Lenaola decried that lawyers were filing submissions drafted with AI in which, as he described it, the language was beautiful, but some authorities, when a judge checked, had been hallucinated by the machine. His instruction was blunt — until the Judiciary issues guidelines, advocates should avoid the tool. Judges do not issue warnings about remote risks.

The hallucination is not a malfunction. It is how the tool works. A large language model (LLM), the most common class of AI systems such as ChatGPT, drafts texts by predicting the most probable continuation of that piece of text. This probability machine can produce a filing that is fluent, correctly formatted, and indistinguishable on the page from competent work. However, the LLM cannot perform the act the advocate’s signature certifies. As a fundamentally stochastic system, a bare LLM has no means of knowing whether the authorities it cites exist: no matter how well it has been trained, if you ask it for a fact, it returns the most probable shape of one. Producing a citation that looks correct is not the same as locating one that is.

The courts have not waited for those guidelines. Submissions drafted by the machine have already been struck out, and judgments resting on them set aside. The principle required no new statute: a document that no one has drawn is a document for which no one is answerable. The shape of the failure is always the same: work is delegated to the machine, the output appears plausible, and no one verifies it before the signature goes down.

That is the heart of it. An advocate’s duty to be candid with the court, and not to mislead it, predates the machine and is unaffected by it. The defence such a filing would otherwise require — that the AI erred, and not the advocate — is one the profession cannot entertain. The system is no officer of the court: it swore no oath, holds no place on the Roll, and is owed no duty by the court because it can be subject to none. Responsibility cannot be assigned to an instrument that cannot be held to account. It cannot be delegated.

It can, however, be enforced — and here I should declare an interest, since the subject of this piece is candour. I am not an advocate. I am a scientist who builds AI systems at Haki, which builds legal AI for Kenya. I met this failure first in my own field. Hallucination is no gentler in science than in law — a fabricated citation in a research paper does the same work as a fabricated authority in a filing, and an LLM will invent a study as readily as it invents a case. During my doctoral research I built a discipline to make that impossible; and because the failure is identical, a discipline that defeats it in one field transfers directly to the other.

The system is called Ledger, and its rule is that no assertion may reach a page unless its exact words can be traced, on the record, to the primary source, and a machine, not the writer’s good intentions, refuses anything that misses that mark. What survives is auditable: another party can follow every claim back to its source. Ledger is provenance made mechanical, so that the one thing a machine should never do, invent a source, becomes the one thing it is not able to do. Before incorporating Ledger into our existing tools, I am submitting it to the Future of Life Foundation for an independent technical evaluation, in competition with other systems built against the same problem of AI hallucinations.

None of this relieves the advocate of anything. The signature remains hers, the judgment remains hers, the duty remains hers. A tool can make it impossible for an AI to file a fiction, but it cannot make advocates care that they do not. The machine cannot be struck off, because it has nothing to lose.

The advocate can, because she has everything to lose: her name, her standing, her reputation. Everything.

Published in The Advocate Magazine Vol 1 Issue 18 – Annual Conference Edition 2026