Can AI learn to fake a clinical trial?
A doctor faked hundreds of stroke trial patients in the 1990s. Now AI can invent them too. Here’s what mathematics can and can’t do about it.
Look at a spreadsheet of real-world numbers and you might expect the leading digits to be spread evenly, with approximately 11% starting with a one, 11% with a two, and so on. Benford’s law says otherwise. In suitable datasets, digit one appears as the first digit about 30% of the time and nine a mere 5%. The law does not govern every dataset, but the underlying idea is useful. Numbers generated by realworld measurements leave patterns that are hard to forge. This is especially true in biology, where every measurement comes with its share of noise.
Numbers generated by real-world measurements leave patterns that are hard to forge
As International Fraud Awareness Week begins on November 15th, one clinical trial scandal shows how mathematics can expose fabricated medical research, and why artificial intelligence (AI) could make fake clinical trials more than just a thought experiment.
The Second European Stroke Prevention Study (ESPS2) ran from 1989 to 1995. It tested whether aspirin and dipyridamole, alone or together, could prevent another stroke in people who had already suffered a stroke or a mini-stroke. Answering such a question demands numbers. ESPS2 was a unique, large and international trial with 7,040 patients randomised across 60 centres in 13 countries. Each patient was assigned to aspirin, dipyridamole, both drugs, or a placebo. The results were published in the Journal of the Neurological Sciences in 1996 and showed that the combination worked best. Among the 6,602 patients in the final efficacy analysis, aspirin plus dipyridamole combo cut the risk of stroke by thirty-seven per cent relative to placebo. ESPS2 provided further support for using aspirin for secondary stroke prevention.
The earliest public disclosure of the scandal was a single line in the trial publication, noting that one centre’s data was dropped because its reliability could not be established. That centre, number 2013, in Almelo in the Netherlands, had reported 438 participants, making it one of the biggest recruiters. Yet at least 349 of them were confirmed in court never to have taken part, and nearly all of those interviewed said they had never enrolled. The neurologist who ran the centre had taken the identities of real stroke patients from his own clinic and used them to create false trial records. These patients knew nothing of the study, had given no consent and received none of the drugs.
Centre 2013 reported serious adverse events but not a single non-serious one. Everywhere else, patients grumbled of the usual side-effects, including the headaches that dipyridamole is known to cause. A site claiming to have treated 438 patients without any non-serious adverse event was, to put it mildly, unusual.
AI can produce a plausible table, but last time I checked, it cannot produce a vial, a signed consent form, or a person who remembers being treated.
The biological measurements were also suspect. Blood pressure readings and lab results varied far less than they should, both between patients and within the same patient across visits. Some followup assessments were logged on weekends, even though the study prohibited assessments on those days. The blood samples provided the most compelling evidence, ultimately delivering the final blow to the credibility of Centre 2013. Drug concentrations from “patients” in this centre produced results that were simply incompatible with the prescribed treatment.
The original investigators grew suspicious in June 1992 and audited the centre seven months later. Fraud was confirmed in June 1993, when analysis of plasma proteins showed that all the blood samples from the centre were a pooled mixture drawn from just a few individuals.
In 2021, researchers revisited the trial database with statistical-monitoring software designed to spot oddities across trial sites. They ran 838 statistical tests on each centre, probing everything from adverse-event reporting and lab results to missing data, patient measurements and correlations between variables. The results were condensed into a single measure called the Data Inconsistency Score.
Centre 2013 ranked as the second most unusual of the 60 centres. If its data followed the patterns the software assumes, a result this extreme would occur about once in 14,000 trials. The score measures how unusual the data is, not how likely it is that someone committed fraud. Surprisingly, another centre scored higher, but there was a legitimate explanation for this site, as it had recruited an unusually old and medically complex patient population. This is why statistical monitoring is best seen as a means to an end when finding sites for further investigation.
Centre 2013 would have been flagged once about a quarter of its data was reported, as early as May 1991, thirteen long months before the trial team grew suspicious. However, we must remember that this was a retrospective analysis (though blinded), conducted after the fraud had been established. The software was never used during ESPS2, so no one knows whether it could have triggered an earlier investigation in real time.
This is where AI introduces a new challenge.
Last year, a study in JAMA Ophthalmology tested whether a large language model could produce fake medical data good enough to survive forensic examination. The authors asked GPT-4o to conjure up datasets for three fictional eye-disease studies, complete with demographics, measurements and complications.
Many chi-square and linear correlation tests later, the conclusion was that 34% of forensic evaluations flagged the dataset as likely fabricated. Some of the slips even resembled those in the ESPS2 case. Patients were attending appointments at weekends, names weren’t matching the recorded sex, ages that were miscalculated and clinical variables that barely correlated when the science says they should have moved together.
Using these findings, the authors designed a custom GPT, with instructions to avoid the mistakes from the first run. The share of tests yelling fabrication fell to 5%, and four of the twelve refined datasets passed as authentic.
It’s worth pointing out that the datasets were deliberately simple and the model received human input when distributing study variables. A real clinical trial typically requires dozens of biological variables across hundreds of patients and multiple trial centres. Those correlations may be harder for synthetic data generators to get right, and medical knowledge could be used to challenge data that passes statistical monitoring, if needed.
And thus, until now, faking a clinical trial would have required a lot of time, effort and statistical gymnastics, effectively discouraging any attempts at the matter. An autonomous language model may not be so easily discouraged. It could run the standard tests on its own output and fix what fails.
Yet, the story of Centre 2013 is one about humans. The dots were connected through a combination of data and interviews in which patients confirmed they had never enrolled. AI can produce a plausible table, but last time I checked, it cannot produce a vial, a signed consent form, or a person who remembers being treated.
The industry’s current rulebook asks sponsors to match oversight to risk.