Bad Science

to the MRC with positive proposals’.

Meanwhile Dr Arthur Krigsman, paediatric gastrointestinal consultant at the New York School of Medicine, was telling hearings in Washington that he had made all kinds of interesting findings in the bowels of autistic children, using endoscopes. This was lavishly reported in the media. Here is the Daily Telegraph:

Scientists in America have reported the first independent corroboration of the research findings of Dr Andrew Wakefield. Dr Krigsman’s discovery is significant because it independently supports Dr Wakefield’s conclusion that a previously unidentified and devastating combination of bowel and brain disease is afflicting young children – a claim that the Department of Health has dismissed as ‘bad science’.

To the best of my knowledge – and I’m pretty good at searching for this stuff – Krigsman’s new scientific research findings which corroborate Andrew Wakefield’s have never been published in an academic journal: certainly there is no trace of them on Pubmed, the index of nearly all medical academic articles.

In case the reason why this is important has not sunk in, let me explain again. If you visit the premises of the Royal Society in London, you’ll see its motto proudly on display: ‘Nullius in verba’ – ‘On the word of no one’. What I like to imagine this refers to, in my geeky way, is the importance of publishing proper scientific papers if you want people to pay attention to your work. Dr Arthur Krigsman has been claiming for years now that he has found evidence linking MMR to autism and bowel disease. Since he hasn’t published his findings, he can claim them until he’s blue in the face, because until we can see exactly what he did, we can’t see what flaws there may be in his methods. Maybe he didn’t select the subjects properly. Maybe he measured the wrong things. If he doesn’t write it up formally, we can never know, because that is what scientists do: write papers, and pull them apart to see if their findings are robust.

Krigsman and others’ failures to publish in peer-reviewed academic journals weren’t isolated incidents. In fact it’s still going on, years later. In 2006, exactly the same thing was happening again. ‘US Scientists Back Autism Link to MMR’, squealed the Telegraph. ‘Scientists Fear MMR Link to Autism’, roared the Mail. ‘US Study Supports Claims of MMR Link to Autism’, croaked The Times a day later.

What was this frightening new data? These scare stories were based on a poster presentation, at a conference yet to occur, on research not yet completed, by a man with a track record of announcing research that never subsequently appears in an academic journal. In fact, astonishingly, four years later, it was Dr Arthur Krigsman again. The story this time was different: he had found genetic material (RNA) from vaccine-strain measles virus in some gut samples from children with autism and bowel problems. If true, this would fit with Wakefield’s theory, which by 2006 was lying in tatters. We might also mention that Wakefield and Krigsman are doctors together at Thoughtful House, a private autism clinic in the USA offering eccentric treatments for developmental disorders.

The Telegraph went on to explain that Krigsman’s most recent unpublished claim was replicating similar work from 1998 by Dr Andrew Wakefield, and from 2002 by Professor John O’Leary. This was, to say the least, a mis-statement. There is no work from 1998 by Wakefield which fits the Telegraph’s claim – at least not in PubMed that I can find. I suspect the newspaper was confused about the infamous Lancet paper on MMR, which by 2004 had already been partially retracted.

There are, however, two papers suggesting that traces of genetic material from the measles virus have been found in children. They have received a mountain of media coverage over half a decade, and yet the media have remained studiously silent on the published evidence suggesting that they were false positives, as we will now see.

One is from Kawashima et al. in 2002, also featuring Wakefield as an author, in which it is claimed that genetic material from measles vaccine was found in blood cells. Doubt is cast on this both by attempts to replicate it, showing where the false positives probably appeared, and by the testimony of Nick Chadwick, the PhD student whose work we described above. Even Andrew Wakefield himself no longer relies on this paper.

The other is O’Leary’s paper from 2002, also featuring Wakefield as an author, which produced evidence of measles RNA in tissue samples from children. Futher experiments, again, have illustrated where the false positives seem to have arisen, and in 2004, when Professor Stephen Bustin was examining the evidence for the legal aid case, he explained how he established to his satisfaction – during a visit to the O’Leary lab – that these were false positives due to contamination and inadequate experimental methods. He has shown, firstly, that there were no ‘controls’ to check for false positives (contamination is a huge risk when you are looking for minuscule traces of genetic material, so you generally run ‘blank’ samples to make sure they do come out blank); he found calibration problems with the machines; problems with log books; and worse. He expanded on this at enormous length in a US court case on autism and vaccines in 2006. You can read his detailed explanation in full online. To my astonishment not one journalist in the UK has ever bothered to report it.

Both of these papers claiming to show a link received blanket media coverage at the time, as did Krigsman’s claims.

What they didn’t tell you

In the May 2006 issue of the Journal of Medical Virology there was a very similar study to the one described by Krigsman, only this one had actually been published, by Afzal et al. It looked for measles RNA in children with regressive autism after MMR vaccination, much like the unpublished Krigsman study, and it used tools so powerful they could detect measles RNA down to single-figure copy numbers. It found no evidence of the magic vaccinestrain measles RNA to implicate MMR. Perhaps because of that unfrightening result, the study was loudly ignored by the press.

Because it has been published in full, I can read it, and pick holes in it, and I am more than happy to do so: because science is about critiquing openly published data and methodologies, rather than press-released chimeras, and in the real world all studies have some flaws, to a greater or lesser extent. Often they are practical ones: here, for example, the researchers couldn’t get hold of the tissue they ideally would have used, because they could not get ethics committee approval for intrusive procedures like lumbar punctures and gut biopsies on children (Wakefield did manage to obtain such samples, but he is, we should remember, currently going through a GMC professional conduct hearing over the issue).

Surely they could have borrowed some existing samples, from children said to be damaged by vaccines? You’d have thought so. They report in the paper that they tried to ask anti-MMR researchers – if that’s not an unfair term – whether they could borrow some of their tissue samples to work on. They were ignored.

Afzal et al. was not reported in the media, anywhere at all, except by me, in my column.

This is not an isolated case. Another major paper was published in the leading academic journal Pediatrics a few months later – to complete media silence – again suggesting very strongly that the earlier results from Kawashima and O’Leary were in error, and false positives. D’Souza et al. replicated the earlier experiments very closely, and in some respects more carefully: most importantly, it traced out the possible routes by which a false positive could have occurred, and made some astonishing findings.

False positives are common in PCR, because it works by using enzymes to replicate RNA, so you start with a small amount in your sample, which is then ‘amplified up’, copied over and over again, until you have enough to measure and work with. Beginning with a single molecule of genetic material, PCR can generate 100 billion similar molecules in an afternoon. Because of this, the PCR process is exquisitely sensitive to contamination – as numerous innocent people languishing in jail could tell you – so you have to be very careful, and clean up as you go.

As well as raising concerns about contamination, D’Souza also found that the O’Leary method might have accidentally amplified the wrong bits of RNA.

Let’s be clear: this is absolutely not about criticising individual researchers. Techniques move on, results are sometimes not replicable, and not all double-checking is practical (although Bustin’s testimony is that standards in the O’Leary lab were problematic). But what is striking is that the media rabidly picked up on the original frightening data, and then completely ignored the new reassuring data. This study by D’Souza, like Afzal before it, was unanimously ignored by the media. It was covered, by my count, in: my column; one Reuters piece which was