Bad Science

Breakfast, Horizon, BBC News, GMTV, London Tonight, Sky News, CBS News in America, The Late Late Show in Ireland, and many more. According to the British media establishment, Professor Patrick Holford is one of our leading public intellectuals: not a vitamin-pill salesman working in the $50-billion food-supplement industry – a fact which is very rarely mentioned, if ever – but an inspiring academic, embodying a diligent and visionary approach to scientific evidence. Let us see what calibre of work is required for journalists to accord you this level of authority in front of the nation.

AIDS, cancer and vitamin pills

I first became fully aware of Holford in a bookshop in Wales. It was a family holiday, I had nothing to write about, and it was New Year. Like a lifesaver, here was a copy of his New Optimum Nutrition Bible, the 500,000-copy best-seller. I seized it hungrily, and looked up the big killers. First I found a section heading which explains that ‘people who take vitamin C live four times longer with cancer’. Excellent stuff.

I looked up AIDS (this is what I call ‘the AIDS test’). Here is what I found on page 208: ‘AZT, the first prescribable anti-HIV drug, is potentially harmful, and proving less effective than vitamin C.’ Now, AIDS and cancer are very serious issues indeed. When you read a dramatic claim like Holford’s, you might assume it’s based on some kind of study, perhaps where people with AIDS were given vitamin C. There’s a little superscript ‘23’, referring you to a paper by someone called Jariwalla. With bated breath I grabbed a copy of this paper online.

The first thing I noticed was that this paper does not mention ‘AZT’. It does not compare AZT with vitamin C. Nor does it involve any human beings: it’s a laboratory study, looking at some cells in a dish. Some vitamin C was squirted onto these cells, and a few complicated things were measured, like ‘giant cell syncytia formation’, which changed when there was lots of vitamin C swimming around. All well and good, but this laboratory-bench finding very clearly does not support the rather dramatic assertion that ‘AZT, the first prescribable anti-HIV drug, is potentially harmful, and proving less effective than vitamin C.’ In fact, it seems this is yet another example of that credulous extrapolation from preliminary laboratory data to clinical claim in real human beings that we have come to recognise as a hallmark of the ‘nutritionist’.

But it gets more interesting. I casually pointed all this out in a newspaper article, and Dr Raxit Jariwalla himself appeared, writing a letter to defend his research paper against the accusation that it was ‘bad science’. This, to me, raised a fascinating question, and one which is at the core of this issue of ‘referenciness’. Jariwalla’s paper was a perfectly good one, and I have never said otherwise. It measured some complicated changes at a basic biological level in some cells in a dish on a lab bench, when they had lots of vitamin C squirted onto them. The methods and results were impeccably well described by Dr Jariwalla. I have no reason to doubt his clear description of what he did.

But the flaw comes in the interpretation. If Holford had said: ‘Dr Raxit Jariwalla found that if you squirt vitamin C onto cells in a dish on a lab bench it seems to change the activity of some of their components,’ and referenced the Jariwalla paper, that would have been fine. He didn’t. He wrote: ‘AZT, the first prescribable anti-HIV drug, is potentially harmful, and proving less effective than vitamin C.’ The scientific research is one thing. What you claim it shows – your interpretation – is entirely separate. Holford’s was preposterous overextrapolation.

I would have thought this was the point at which many people might have said: ‘Yes, in retrospect, that was perhaps a little foolishly phrased.’ But Professor Holford took a different tack. He has claimed that I had quoted him out of context (I did not: you can view the full page from his book online). He has claimed that he has corrected his book (you can read about this in a note at the back of the book you are holding). He has thrown around repeated accusations that I have only criticised him on this point because I am a pawn of big pharmaceutical corporations (I am not; in fact, bizarrely, I am one of their most vicious critics). Crucially, he suggested that I had focused on a trivial, isolated error.

A vaguely systematic review

The joy of a book is that you have plenty of space to play with. I have here my copy of The New Optimum Nutrition Bible. It’s ‘the book you have to read if you care about your health’, according to the Sunday Times quote on the front cover. ‘Invaluable’, said the Independent on Sunday, and so on. I have decided to check every single reference, like a crazed stalker, and I will now dedicate the entire second half of this book to producing an annotated edition of Holford’s weighty tome.

Only kidding.

There are 558 pages of plausible technical jargon in Holford’s book, with complicated advice on what foods to eat, and which kinds of pills you should buy (in the ‘resources’ section it turns out that his own range of pills are ‘the best’). For our sanity I have restricted our examination to one key section: the chapter where he explains why you should take supplements. Before we begin, we must be very clear: I am only interested in Professor Holford because he teaches the nutritionists who treat the nation, and because he has been given a professorship at Teesside University, with plans for him to teach students and supervise research. If Professor Patrick Holford is a man of science, and an academic, then we should treat him as one, with a scrupulously straight bat.

So, turning to Chapter 12, page 97 (I’m working from the ‘completely revised and updated’ 2004 edition, reprinted in 2007, if you’d like to follow the working at home), we can begin. You’ll see that Holford is explaining the need to eat pills. This might be an apposite moment to mention that Professor Patrick Holford currently has his own range of best-selling pills, at least twenty different varieties, all featuring a photograph of his smiling face on the label. This range is available through the pill company BioCare, and his previous range, which you will see in older books, was sold by Higher Nature.

My whole purpose in writing this book is to teach good science by examining the bad, so you will be pleased to hear that the very first claim Holford makes, in the very first paragraph of his key chapter, is a perfect example of a phenomenon we have already encountered: ‘cherry-picking’, or selecting the data that suits your case. He says there is a trial which shows that vitamin C will reduce the incidence of colds. But there is a gold-standard systematic review from Cochrane which brings together the evidence from all twenty-nine different trials on this subject, covering 11,000 participants in total, and concluded that there is no evidence that vitamin C prevents colds. Professor Holford doesn’t give a reference for his single, unusual trial which contradicts the entire body of research meticulously summarised by Cochrane, but it doesn’t matter: whatever it is, because it conflicts with the meta- analysis, we can be clear that it is cherry-picked.

Holford does give a reference, immediately afterwards, for a study where blood tests showed that seven out of ten subjects were deficient in vitamin B. There is an authoritative-looking superscript number in the text. Turning to the back of the book, we find that his reference for this study is a cassette you used to be able to buy from his own Institute for Optimum Nutrition (it’s called The Myth of the Balanced Diet). We then have a twenty-five-year-old report from the Bateman Catering Organisation (who?), apparently with the wrong date; a paper on vitamin B12; some ‘experiment’ without a control reported in a 1987 ION pamphlet so obscure it’s not even in the British Library (which has everything). Then there is a bland statement referenced to an article in the Institute for Optimum Nutrition’s Optimum Nutrition Magazine, and an uncontroversial claim supported by a valid paper – the children of mothers who have taken folic acid during pregnancy have fewer birth defects, a well-established fact reflected in Department of Health guidelines – because there has to be a grain of commonsense truth in the spiel somewhere. Getting back to the action, we are told about a study on ninety schoolchildren who get 10 per cent higher IQ scores after taking a high-dose multivitamin pill, sadly without a reference, before a true gem: one paragraph with four references.

The first is to a study by the great Dr R.K. Chandra, a disgraced researcher whose papers have been discredited and retracted, who has been the subject of major articles on research fraud, including one by Dr Richard Smith in the British Medical Journal called ‘Investigating the previous studies of a fraudulent author’. There is an entire three-part investigative documentary series on his worrying career made by Canada’s CBC (you can watch it online), and at the conclusion of it he was, to all intents and purposes, in hiding in India. He has 120 different bank accounts in various tax havens, and he did, of course, patent his own multivitamin mixture, which he sells as an ‘evidencebased’ nutrition supplement for the elderly. The ‘evidence’ is largely derived from his own clinical trials.

In the name of scrupulous fairness, I am happy to clarify that much of this has come out since the first edition of Holford’s book; but there had been serious questions about Chandra’s research for some time, and