Bad Science

might have come about, the world would like to hear from you. Follow the reference, read the full paper online and start a blog, or write a letter to the journal that published it.

The Nonsense du Jour

Now we need to raise our game. Food has become, without question, a national obsession. The Daily Mail in particular has become engaged in a bizarre ongoing ontological project, diligently sifting through all the inanimate objects of the universe in order to categorise them as a cause of – or cure for – cancer. At the core of this whole project are a small number of repeated canards, basic misunderstandings of evidence which recur with phenomenal frequency.

Although many of these crimes are also committed by journalists, we will be reviewing them later. For the moment we will focus on ‘nutritionists’, members of a newly invented profession who must create a commercial space to justify their own existence. In order to do this, they must mystify and overcomplicate diet, and foster your dependence upon them. Their profession is based on a set of very simple mistakes in how we interpret scientific literature: they extrapolate wildly from ‘laboratory bench data’ to make claims about humans; they extrapolate from ‘observational data’ to make ‘intervention claims’; they ‘cherry-pick’; and, lastly, they quote published scientific research evidence which seems, as far as one can tell, not to exist.

It’s worth going through these misrepresentations of evidence, mainly because they are fascinating illustrations of how people can get things wrong, but also because the aim of this book is that you should be future-proofed against new variants of bullshit. There are also two things we should be very clear on. Firstly, I’m picking out individual examples as props, but these are characteristic of the genre; I could have used many more. Nobody is being bullied, and none of them should be imagined to stand out from the nutritionist crowd, although I’m sure some of the people covered here won’t be able to understand how they’ve done anything wrong.

Secondly, I am not deriding simple, sensible, healthy eating advice. A straightforwardly healthy diet, along with many other aspects of lifestyle (many of which are probably more important, not that you’d know it from reading the papers) is very important. But the media nutritionists speak beyond the evidence: often it is about selling pills; sometimes it is about selling dietary fads, or new diagnoses, or fostering dependence; but it is always driven by their desire to create a market for themselves, in which they are the expert, whereas you are merely bamboozled and ignorant.

Prepare to switch roles.

The four key errors

Does the data exist?

This is perhaps the simplest canard of all, and it happens with surprising frequency, in some rather authoritative venues. Here is Michael van Straten on BBC Newsnight, talking ‘fact’. If you prefer not to take it on faith that his delivery is earnest, definitive, and perhaps even slightly patrician, you can watch the clip online.

‘When Michael van Straten started writing about the magical medicinal powers of fruit juices, he was considered a crank,’ Newsnight begins. ‘But now he finds he’s at the forefront of fashion.’ (In a world where journalists seem to struggle with science, we should note that Newsnight has ‘crank’ at one end of the axis, and ‘fashion’ at the other. But that chapter comes later.) Van Straten hands the reporter a glass of juice. ‘Two years added to your life expectancy in that!’ he chuckles – then a moment of seriousness: ‘Well, six months, being honest about it.’ A correction. ‘A recent study just published last week in America showed that eating pomegranates, pomegranate juice, can actually protect you against ageing, against wrinkles,’ he says.

Hearing this on Newsnight, the viewer might naturally conclude that a study has recently been published in America showing that pomegranates can protect against ageing. But if you go to Medline, the standard search tool for finding medical academic papers, no such study exists, or at least not that I can find. Perhaps there’s some kind of leaflet from the pomegranate industry doing the rounds. He goes on: ‘There’s a whole group of plastic surgeons in the States who’ve done a study giving some women pomegranates to eat, and juice to drink, after plastic surgery and before plastic surgery: and they heal in half the time, with half the complications, and no visible wrinkles!’ Again, it’s a very specific claim – a human trial on pomegranates and surgery – and again, there is nothing in the studies database.

So could you fairly characterise this Newsnight performance as ‘lying’? Absolutely not. In defence of almost all nutritionists, I would argue that they lack the academic experience, the ill-will, and perhaps even the intellectual horsepower necessary to be fairly derided as liars. The philosopher Professor Harry Frankfurt of Princeton University discusses this issue at length in his classic 1986 essay ‘On Bullshit’. Under his model, ‘bullshit’ is a form of falsehood distinct from lying: the liar knows and cares about the truth, but deliberately sets out to mislead; the truth-speaker knows the truth and is trying to give it to us; the bullshitter, meanwhile, does not care about the truth, and is simply trying to impress us:

It is impossible for someone to lie unless he thinks he knows the truth. Producing bullshit requires no such conviction … When an honest man speaks, he says only what he believes to be true; and for the liar, it is correspondingly indispensable that he considers his statements to be false. For the bullshitter, however, all these bets are off: he is neither on the side of the true nor on the side of the false. His eye is not on the facts at all, as the eyes of the honest man and of the liar are, except insofar as they may be pertinent to his interest in getting away with what he says. He does not care whether the things he says describe reality correctly. He just picks them out, or makes them up, to suit his purpose.

I see van Straten, like many of the subjects in this book, very much in the ‘bullshitting’ camp. Is it unfair for me to pick out this one man? Perhaps. In biology fieldwork, you throw a wired square called a ‘quadrat’ at random out onto the ground, and then examine whatever species fall underneath it. This is the approach I have taken with nutritionists, and until I have a Department of Pseudoscience Studies with an army of PhD students doing quantitative work on who is the worst, we shall never know. Van Straten seems like a nice, friendly guy. But we have to start somewhere.

Observation, or intervention?

Does the cock’s crow cause the sun to rise? No. Does this light switch make the room get brighter? Yes. Things can happen at roughly the same time, but that is weak, circumstantial evidence for causation. Yet it’s exactly this kind of evidence that is used by media nutritionists as confident proof of their claims in our second major canard.

According to the Daily Mirror, Angela Dowden, RNutr, is ‘Britain’s Leading Nutritionist’, a monicker it continues to use even though she has been censured by the Nutrition Society for making a claim in the media with literally no evidence whatsoever. Here is a different and more interesting example from Dowden: a quote from her column in the Mirror, writing about foods offering protection from the sun during a heatwave: ‘An Australian study in 2001 found that olive oil (in combination with fruit, vegetables and pulses) offered measurable protection against skin wrinkling. Eat more olive oil by using it in salad dressings or dip bread in it rather than using butter.’

That’s very specific advice, with a very specific claim, quoting a very specific reference, and with a very authoritative tone. It’s typical of what you get in the papers from media nutritionists. Let’s go to the library and fetch out the paper she refers to (‘Skin wrinkling: can food make a difference?’ Purba MB et al. J Am Coll Nutr. 2001 Feb; 20(1): 71–80). Before we go any further, we should be clear that we are criticising Dowden’s interpretation of this research, and not the research itself, which we assume is a faithful description of the investigative work that was done.

This was an observational study, not an intervention study. It did not give people olive oil for a time and then measure differences in wrinkles: quite the opposite, in fact. It pooled four different groups of people to get a range of diverse lifestyles, including Greeks, Anglo-Celtic Australians and Swedish people, and it found that people who had completely different eating habits – and completely different lives, we might reasonably assume – also had