In 2003 a Belgian election anomaly investigation shrugged. In 2017 it’s described as cosmic rays hacking our elections. Ridiculous.
On Friday 17 February 2017, Bharat Bhuva, a professor at Vanderbilt University, gave a speech at the annual meeting of the American Association for the Advancement of Science in Boston. The University, in a desperate attempt to make this sound interesting, runs an irresponsible headline saying “Alien particles from outer space are wreaking low-grade havoc on personal electronic devices“. News outlets from The Independent, to Gizmodo, to Motherboard, and more all carry this tantalising headline. Extraordinary claims require extraordinary evidence. But journalists lapped this up and reported it, failing to notice that there was no evidence at all.
Absence of Evidence is not Evidence of Absence
To be fair, Professor Bhuva says a very reasonable thing: “The only way you can determine that it is a single-event upset is by eliminating all the other possible causes.” That’s true. The problem is he does not systematically eliminate any other possible causes. He concludes that bits flip in hardware when there’s no good evidence that it was not software. Having concluded that bits flip in hardware, he concludes that the cause is cosmic rays. He’s not giving you all the reasons a bit might flip in hardware and concluding that cosmic rays are overwhelmingly more likely. He’s only talking about the probabilities that cosmic rays do it, without referencing the probabilities of any other cause of a bit flip.
Cosmic Rays Did Not Change the Schaerbeek Election
The juicy hook for journalists, especially with Trump moaning about election hacking, is this supposed story about Schaerbeek, Belgium and their election in 2003 that had a supposed bit flip. Professor Bhuva would have you believe that this was cosmic rays affecting a voting machine and impacting the election. Here’s the actual research on the incident.
A quick bit of googling turns up this discussion in 2004 on comp.risks. In it, we find references to original sources, including the Rapport concernant les élections du 18 mai 2003 (in French) and some commentary from the Association Electronique Libre. People were skeptical of the “random bit flip” theory even then.
The Belgians Shrugged
A team of experts looked into what happened. In an official report, translated by Google, they write “Since no error was found in the software …the error was most likely caused by a spontaneous and random inversion of a binary position”. I’ve personally spent months of my life doing source code review on voting machines. Anyone who knows anything about software testing or code review knows what a difficult task it is. To conclude that a code review found nothing, therefore the code is flawless is ridiculous. But even then, they’re just saying “we couldn’t find anything in the software, so maybe it was a hardware bug because we’ve heard of that happening.”
That shrug becomes a definite hardware bit flip to Professor Bhuva. He has concluded that there is no way it was a software bug that was missed. He is sure it was a hardware error. Moreover, he has somehow “eliminated all the other possible causes” of bit flips—a power surge, a defect in the manufacturing, or something mundane like that. Not only does he conclude that the bit flip happened in hardware, he concludes that the bit flip was caused by cosmic rays. The evidence appears to have been gathered through the time-honoured technique called DRE.¹
Let’s Keep Talking Probabilities
Much of Professor Bhuva’s remarks talk about probabilities of particles and the likelihood that particles will have an effect on a computer because of the density of memory and so on. Accepting that a single bit will flip in hardware, what is the probability that a single bit flip in a machine, will produce an ordinary error like this? Think of the billions and billions of bits that can be flipped: hardware chips like disk controllers, CPUs, RAM, PCI bus controllers and zillions of other components in one voting machine. Even if we focus only on things stored in RAM, what are the chances that flipping a single bit will allow anything to continue running? Most of your operating system and application software will either crash badly, corrupt data somewhere, or will ignore the bit flip entirely. But the chances of flipping such a meaningful bit: one of a handful of bits in one part of the software that would have this effect? That’s as astronomically unlikely. So we have an astronomically unlikely event: a cosmic ray causing a bit flip, compounded with another astronomically unlikely event: flipping only a single bit in such a meaningful way.
Extraordinary Claims Require Extraordinary Evidence
Carl Sagan popularised this phrase. Bhuva’s claim is beyond extraordinary, but his “evidence” is little more than “this might be what happened.” It’s the worst kind of correlation/causation confusion. “Bits flip due to cosmic rays. The post mortem analysts think a bit might have flipped. Therefore cosmic rays flipped a single, really important bit in the voting machine.” The 13th bit could have flipped due to an integer overflow, a hardware anomaly of any ordinary sort, or even some bug in a floppy disk (the Schaerbeek voting system used floppy disks, which themselves have a nice, high error rate).
As a software professional who has spent a career hunting hard-to-find bugs, it offends me to say “these guys couldn’t find any software bugs, so we’re gonna chalk it up to cosmic rays.” I appreciate that journalists will struggle to criticise this kind of science, especially in the face of a university press team pushing the story. But this is such balderdash that it really needs to be retracted.
¹ DRE: direct rectal extraction