South Africa's Draft National Artificial Intelligence Policy was published in the Government Gazette on 10 April 2026. On 12 June it was withdrawn in its entirety.
The reason was that at least six of its sixty-seven academic references did not exist. Editors at the South African Journal of Philosophy, AI & Society and the Journal of Ethics and Social Philosophy confirmed that articles attributed to their publications had never been published in them. The citations had authors, titles and journals. They did not have papers.
The obvious framing is the ironic one: the country's AI policy was written with AI, badly. That framing is comfortable and it is a distraction. Every organisation reading this has a process that could produce the same outcome, and most of them believe otherwise.
The failure was not the hallucination
A generative model producing a plausible citation that does not exist is not a malfunction. It is the expected behaviour of a system that predicts likely text. Anyone who has used one knows this. Treating it as the story is like blaming a calculator for a number nobody checked.
The actual failure has a location, and it is downstream. A document moved from drafting, through internal review, through Cabinet approval on two separate occasions, into the Government Gazette, and out for public comment. At each of those stages a person or a committee signed something. None of that machinery caught six invented sources in a sixty-seven-item bibliography — something a research assistant with a browser would have found in an afternoon.
What was missing was not technology. It was a named person whose job it was to be able to say I checked this, and here is what I checked.
Why review processes fail exactly here
AI-generated text fails review differently from human error, and organisations are not yet calibrated for it.
Human error is usually legible. A person who has not read a paper tends to cite it vaguely, or get the year wrong, or describe it in terms that do not quite fit. The mistake carries a signature. Reviewers have spent careers learning to read those signatures.
A fabricated citation has no signature. It is perfectly formatted, plausibly titled, attributed to a real journal in the right field, and consistent with the argument it supports. It looks exactly like a citation from someone who did the reading. Every heuristic a reviewer has developed for spotting sloppiness returns clean, because the output is not sloppy. It is confident and wrong.
This is the specific hazard: AI output degrades gracefully in appearance and catastrophically in substance. Review processes built to catch visible carelessness do not catch it, and the people running those processes have no way to know their checks stopped working.
The verification asymmetry
There is a structural problem underneath this that will not go away with better models.
Generating a citation now costs nothing. Verifying one still costs a few minutes of a person's attention. When one side of that equation collapses to zero and the other does not, volume moves to wherever the cost went, and verification becomes the bottleneck it was never designed to be.
A bibliography of sixty-seven references used to imply sixty-seven acts of reading. It no longer implies anything at all. The signal that a citation list carried — someone engaged with this literature — has been severed from the artefact, and nobody has yet rebuilt it.
This is why "a human reviewed it" is doing far less work than the phrase suggests. Reviewed how? Against what? A reviewer who reads for coherence will pass a fabricated citation every time, because fabricated citations are coherent. Only a reviewer who opens the source catches it, and nothing in most review processes requires that or records whether it happened.
What we would have looked for
The question we ask of any organisation is narrow and awkward: for this specific output, who is accountable, and what could they actually produce if challenged?
Applied here, that means:
- Was there a named person responsible for the integrity of the evidence base, as distinct from the policy argument? Not a committee. A person.
- Was verification a defined step with a recorded outcome, or an assumption distributed across everyone who touched the document — which is how it ends up performed by nobody?
- Did AI use survive contact with the review process? If drafters used a model, did the reviewers know? An unmarked AI draft is reviewed as though it were a human one, using heuristics that do not apply.
- Was there a point of refusal? Could someone in that chain have said this is not ready and been heard? Accountability without the power to stop something is decoration.
None of these require sophisticated technology. All of them require someone to have been made responsible in advance, in a way that would be visible afterwards.
The uncomfortable part
The department has since adopted an internal framework governing its own use of AI, two officials are on precautionary suspension, and a revised policy is targeted for Cabinet in November 2026. That is a reasonable response.
But it is worth sitting with what happened rather than moving quickly past it. This was not a fringe document. It was a national policy, drafted by a government department, approved by Cabinet twice, and published in the Gazette. It passed every gate it was supposed to pass. The gates were not fake — they were real processes, staffed by real people, doing what those processes had always done.
They were simply built for a world in which producing sixty-seven plausible references required doing sixty-seven pieces of work. That world ended, and the processes did not notice.
Most organisations are in the same position and have not yet had the news delivered as publicly.