A Statistical Conviction: The Lucia de Berk Story
Introduction
I recently picked up my copy of Sean Carroll's interesting book published in 2016 with the lofty title "The Big Picture: The Origins of Life, Meaning and the Universe Itself" to re-read that for the concept of "poetic naturalism" that it discusses. It's the the kind of skepticism that is very appealing to me. For those unfamiliar with Sean Carroll, he is a well known theoretical physicist and popularizer of general relativity and cosmology. A few chapters into the book, I re-read the story of Lucia de Berk that Carroll adds to the chapter titled "Reasons Why," which tackles an age old philosophical idea given the name 'Principle of Sufficient Reason' by Gottfried Leibniz, who summarized it as "nothing happens without a reason".
This short write up is a synopsis of story of Lucia de Berk and it is a cautionary tale for our times. Her sad story takes on an even deeper meaning now in the age of AI.
The Case Against Lucia de Berk and the Conviction
In September 2001, the unexplainable death of a baby at the JKZ (a children's hospital) in The Hague in the Netherlands prompted the scrutiny of other deaths over the past year, and soon the hospital realized with alarm that many of the deaths and incidents (like resuscitations) were indeed suspicious. They had a serial killer in their midst! The deaths were linked to the shifts of nurse Lucia de Berk by an alert colleague of hers who felt that what happened during Lucia's care just could not be a coincidence any longer, and informed her superiors about her suspicions. The managing director of the hospital who looked at the correlation of the occurrences of deaths and resuscitations with Lucia's shifts could not also escape the conclusion that it was indeed highly suspicious. Soon the hospital armed with a list of unexplainable deaths in the care of nurse de Berk notified the police who promptly began an investigation. A hospital committee was set up to review the deaths and found an unnaturally high numbers of deaths or resuscitations occurred during Lucia's shifts. The hospital also brought in an outside expert to assess if this could in any way be just coincidence. The scrutiny continued to build and soon Lucia was charged with thirteen murders and four attempted murders. The experts during the trial established the improbability of this being coincidence. The medical expert who testified opined that that while each case could look like a sudden infant death, which happens without a clear cause, the statistical chance of one nurse being present at this many occurrences was vanishing small, basically making it exceedingly improbable that these children died of natural causes. A number that emerged during the trial as the odds against this being coincidence was 1 in 342 million, courtesy of a Professor Eiffers. Convinced by such insuperable odds, in June 2004, the Court of Appeal in The Hague convicted Lucia formally for seven murders and three attempted murders and sentenced her to life in prison with no chance of parole.
The Case For Lucia de Berk and the Appeal
Two years after her conviction, in 2006, Professor Ton Derksen, philosopher of science, presented her case (summed up in his now famous book [2] on the case), which combines the inquiry and research into the case by him and his partner Dr. Metta de Noo, to a commission that reviewed such closed cases. The Derksen book systematically annihilates the case against Lucia -- from the blind rush to judgment to the flawed statistics put together by experts to disprove coincidence and to the incredible bias of the Court in finding her guilty.
The Derksen analysis of the incredible odds of 1 to 342 million of this being coincidence reduced it this to a paltry 1 in 44. Other statisticians like Professor Richard Gill who began to question the data brought the odds down to 1 in 9. On average 1 in 9 nurses can face this situation! To look at this in witch hunting terms: a lot of "serial killer nurses" are working in the hospitals of Netherlands!
There was soon a groundswell of emotion against what was now deemed a terrible miscarriage of justice and a petition was made in 2007 to review the case. It is only too clear that the wheels of justice grind slowly. After a lot of legal wrangling, the Supreme Court of the Netherlands finally decided in December 2008 that the case must be reopened and referred it to the Court of Arnhem. Finally on 14th April 2010, nine years and three months after she was taken into custody, Lucia de Berk was acquitted of all charges and set free. What a terrible miscarriage of justice!
"Lucia de Berk never admitted her guilt but was convicted in spite of there being no eye witnesses and no direct incriminating evidence against her. 'Statistics drove the case from start to end,' says the British-born statistician Richard Gill, professor of mathematical statistics at the University of Leiden in the Nertherlands. 'Statistics and psychology. Lucia's conviction for serial murder, and even the 'proof' that there were any murders at all – let alone by whom – were almost entirely based on wrong statistical data, wrongly analysed, and wrongly interpreted.' " (From the Independent: Did Statistics Damn Lucia de Berk? Nigel Hawkes, April 10, 2010)
How Could This Happen?
The Lucia de B website [4] asks the question: "How is it possible that in what is assumed to be a civilized country, somebody is given a life sentence without any concrete evidence?"
The Derksen book analyzes this question in the first chapter of the book and offers "nine points of conviction" the led to this blind rush to judgment, the set up of the witch hunt by summoning flawed science and statistics for that purpose, and the terrible failure of the Court to remain objective. The media circus fanned the flames by painting Lucia as evil and driven by a compulsion to murder. Exculpatory data was also ignored by the prosecution.
(See the synopsis of the case "Fabrication of Facts: The Lure of The Incredible Coincidence" provided by Ton Derksen for the details. [1])
In their paper "Elementary Statistics on Trial (the Case of Lucia de Berk)" by Richard Gill et al [3], the authors present the startling conclusion: "One in twenty-six will go to jail". What they mean is that "a modest amount of heterogeneity turns an almost impossible occurrence into something merely mildly unusual." As Gill et al show in the paper, even a simple base rate calculation of a total 26 unexplainable incidents over 1734 total number of shifts would make an innocent hardworking nurse like Lucia with 203 shifts to her favor experience 3 such incidents. The probability curve produced showed that a nurse could be associated with 5 incidents with a ~25% chance (1 in 4) and 7 incidents with a ~14% chance (~1 in 7), odds that are ordinary occurrences. From the paper: "A modest amount of variation makes the chance that an innocent nurse experiences at least as many incidents as the number Lucia actually did experience, the somewhat unremarkable one in seven. Making some less favourable choices to her in the data cleaning process, only decreases this chance to one in twenty-six." This is a real caution against the naive use of statistics. The case was bedeviled by a faulty model, tenuous assumptions of uniformity and independence, extreme model sensitivity, and spurious correlation.
(For folks interested in the statistical modeling aspects of this sad case, links to the Derksen book synopsis [1] and the Gill et al paper [3] are provided below.)
Lucia de Berk's ordeal is a cautionary tale about scientific rigor and human psychology. We are wired to discern patterns and this remarkable ability can also lead us really astray.
To wrap up, it is exceedingly important when faced with such critical situations to pay attention to the following key factors:
- Blindly rushing to judgment
- Being able to discern a fabricated trial or a witch hunt
- False scientific rationalization
- Falling for a false image
- Understanding the inbuilt bias within systems
Look forward to your comments and feedback.
―Suresh Babu, May 19, 2020
References:
- "The Fabrication of Facts: the Lure of the Incredible Coincidence" -- by Ton Derksen, 2007 -- an English synopsis of his famous Dutch book "Lucia de B: Reconstruction of a Miscarriage of Justice"
- "Lucia de B: Reconstruction of a Miscarriage of Justice" -- by Ton Derksen, 2006
- "Elementary Statistics on Trial (the Case of Lucia de Berk)" -- Gill, Gruneboom, de Jong, June 2010
- Lucia de B website -- an amazing compendium of all the key facts surrounding the Lucia de Berk case