Bad Science

Durham Council and Equazen were so successful in their publicity drive, whether through an uncontainable enthusiasm for a positive result or simple foolishness (I really don’t know which) that they effectively sabotaged their ‘trial’. Before the first fish-oil capsule was swallowed by a single child, the Eye Q branded supplement and trial had received glowing publicity in the local papers, the Guardian, the Observer, the Daily Mail, The Times, Channel 4, the BBC, ITV, the Daily Express, the Daily Mirror, the Sun, GMTV, Woman’s Own, and many more. Nobody could claim that the children weren’t well primed.

You’re not an educational psychologist. You’re not the Head of Education at a council. You’re not the long- standing MD of a multi-million-pound pill business running huge numbers of ‘trials’. But I am quite sure that you understand very clearly all of these criticisms and concerns, because this isn’t rocket science.

Durham defend themselves

Being a fairly innocent and open-minded soul, I went to the people behind the trial, and put it to them that they had done the very things which would guarantee that their trial would produce useless results. That is what anyone would do in an academic context, and this was a trial after all. Their response was simple. ‘We’ve been quite clear,’ said Dave Ford, Chief Schools Inspector for Durham, and the mastermind behind the project to give out the capsules and measure the results. ‘This is not a trial.’

This felt a bit weak. I call up to suggest that they’re doing a badly designed piece of research, and suddenly everything’s OK, because it’s not actually a ‘trial’? There were other reasons for thinking this was a fairly implausible defence. The Press Association called it a trial. The Daily Mail called it a trial. Channel 4 and ITV and everyone covering it all presented it, very clearly, as research (you can see the clips at badscience. net). More importantly, Durham Council’s own press release called it a ‘study’ and a ‘trial’, repeatedly. They were giving something to schoolchildren and measuring the result. Their own descriptive term for this activity was ‘trial’. Now they were saying it wasn’t a trial.

I moved on to Equazen, the manufacturer which is still being lauded throughout the press for its involvement in these ‘trials’ which were almost guaranteed – by virtue of the methodological flaws we have already discussed – to produce spurious positive results. Adam Kelliher, chief executive of the company, clarified further: this was an ‘initiative’. It was not a ‘trial’, nor was it a ‘study’, so I could not critique it as such. Although it was hard to ignore the fact that the Equazen press release talked about giving a capsule and measuring the results, and the word which the company itself used to describe this activity was: ‘trial’.

Dr Madeleine Portwood, the senior educational psychologist running the study, called it a ‘trial’ (twice in the Daily Mail). Every single write-up described it as research. They were giving ‘X’ and measuring change ‘Y’. They called it a trial, and it was a trial – but a stupid trial. Simply saying, ‘Ah, but this is not a trial’ didn’t strike me as an adequate – nor indeed a particularly adult – defence. They didn’t seem to think a trial was even necessary, and Dave Ford explained that the evidence already shows fish oils are beneficial. Let’s see.

The fish-oil evidence

Omega-3 oils are ‘essential fatty acids’. They’re called ‘essential’ because they’re not made by the body (unlike glucose or vitamin D, for example), so you have to eat them. This is true of a lot of things, like many vitamins, for example, and it’s one of the many reasons why it’s a good idea to eat a varied diet, pleasure being another.

They are found in fish oils, and – in slightly different form – in evening primrose oil, linseed oil and other sources. If you look at the flow charts in a biochemistry textbook you will see that there is a long list of functions which these molecules perform in the body: they are involved in constructing membranes, and also some of the molecules that are involved in communication between cells, for example during inflammation. For this reason some people think it might be useful to eat them in larger amounts.

I’m open to the idea myself, but there are good reasons to be sceptical, because there is a lot of history here. In the past, decades before the Durham ‘trials’, the field of essential fatty acid research has seen research fraud, secrecy, court cases, negative findings that have been hushed up, media misreporting on a massive scale, and some very striking examples of people using the media to present research findings direct to the public in order to circumvent regulators. We’ll come back to that later.

There have been – count them – six trials to date on fish oil in children. Not one of these trials was done in ‘normal’ mainstream children: all of them have been done in special categories of children with one diagnosis or another – dyslexia, ADHD, and so on. Three of the trials had some positive findings, in some of the many things they measured (but remember, if you measure a hundred things in a study, a few of them will improve simply by chance, as we will see later), and three were negative. One, amusingly, found that the placebo group did better than the fish-oil group on some measures. They are all summarised online at badscience. net.

And yet: ‘All of our research, both published and unpublished, shows that the Eye Q formula can really help enhance achievement in the classroom,’ says Adam Kelliher, CEO of Equazen. All of it.

To take a statement like this seriously, we would have to read the research. I am not for one nanosecond accusing anybody of research fraud: in any case, if anyone did suspect fraud, reading the research would not help, because if people have faked their results with any enthusiasm then you need a forensic statistician and a lot of time and information to catch them out. But we do need to read published research in order to establish whether the conclusions drawn by the stakeholders in the research are valid, or whether there are methodological problems that make their interpretation the product of wishful thinking, incompetence, or perhaps even simply a judgement call with which you would not concur.

Paul Broca, for example, was a famous French craniologist in the nineteenth century whose name is given to Broca’s area, the part of the frontal lobe involved in the generation of speech (which is wiped out in many stroke victims). Among his other interests, Broca used to measure brains, and he was always rather perturbed by the fact that the German brains came out a hundred grams heavier than French brains. So he decided that other factors, such as overall body weight, should also be taken into account when measuring brain size: this explained the larger Germanic brains to his satisfaction. But for his prominent work on how men have larger brains than women, he didn’t make any such adjustments. Whether by accident or by design, it’s a kludge.

Cesare Lombroso, a nineteenth-century pioneer of ‘biological criminology’, made similarly inconsistent fixes in his research, citing insensitivity to pain among criminals and ‘lower races’ as a sign of their primitive nature, but identifying the very same quality as evidence of courage and bravery in Europeans. The devil is in the detail, and this is why scientists report their full methods and results in academic papers, not in newspapers or on television programmes, and it is why experimental research cannot be reported in the mainstream media alone.

You might feel, after the ‘trial’ nonsense, that we should be cautious about accepting Durham and Equazen’s appraisal of their own work, but I’d be suspicious of the claims from a lot of serious academics in exactly the same way (they would welcome such suspicion, and I’d be able to read the research evidence they were drawing on). I asked Equazen for their twenty positive studies, and was told I would have to sign a confidentiality agreement to see them. That’s a confidentiality agreement to review the research evidence for widely promoted claims, going back years, in the media and by Durham Council employees, about a very controversial area of nutrition and behaviour, of huge interest to the public, and about experiments conducted – forgive me if I’m getting sentimental here – on our schoolchildren. I refused.

Meanwhile, all over the newspapers and television, everywhere you looked, going back to at least 2002, there were reports of positive fish-oil trials in Durham, using Equazen products. There seemed to have been half a dozen of these trials, in various locations, performed by Durham Council staff on Durham state school children, yet there was no sign of anything in the scientific literature (beyond one study by a researcher from Oxford, which was on children with developmental coordination disorder). There were wildly enthusiastic Durham Council press releases that talked about positive trials, sure. There were Madeleine Portwood interviews with the press, in which she talked enthusiastically about the positive results (and talked too about how the fish oil was improving children’s skin conditions, and other problems) – but no published studies.

I contacted Durham. They put me on to Madeleine Portwood, the scientific brains behind this enormous and enduring operation. She appears regularly on telly talking about fish oils, using inappropriately technical terms like ‘limbic’ to a lay audience. ‘It sounds complicated,’ say TV presenters, ‘but the science says …’ Portwood is evidently