very enthusiastic about talking to parents and journalists, but she did not return my calls. The press office took a week to reply to my emails. I asked for details of the studies they had performed, or were performing. The responses appeared to be inconsistent with the media coverage. At least one trial seemed to be missing. I asked for methodological details of the studies they were doing, and results of the ones that were completed. Not until we publish, they said.
Equazen and Durham Council had coached, preened and spoon-fed a huge number of journalists over the years, giving them time and energy; as far as I can see there is only one difference between me and those reporters: from what they wrote, they clearly know very little about trial design, whereas I know, well, a fair bit (and now so do you).
Meanwhile I kept being referred to the durhamtrials.org website, as if this contained useful data. It had evidently bamboozled many journalists and parents before me, and it’s linked to by many news stories and Equazen adverts. But as a source of information about the ‘trials’, this site is a perfect illustration of why you should publish a trial before you make any dramatic claims about the results. It’s hard to tell what’s there. The last time I looked, there was some data borrowed from a proper trial published elsewhere by some Oxford researchers (which happened to be done in Durham), but other than that, no sign of Durham’s own placebo-controlled trials which kept appearing in the news. There were plenty of complicated-looking graphs, but they seem to be on special Durham ‘trials’, with no placebo-control group. They seem to describe improvements, with sciencey-looking graphics to illustrate them, but there are no statistics to say if the changes were statistically significant.
It’s almost impossible to express just how much data is missing from this site, and how useless that renders what is there. As an example, there is a ‘trial’ with its results reported in a graph, but nowhere on the entire site, as far as I can see, does it tell you how many children were in the study reported. It’s hard to think of a single more basic piece of information than that. But you can find plenty of gushing testimonials that wouldn’t be out of place on an alternative therapists’ website selling miracle cures. One child says: ‘Now I am not so interested in the TV. I just like reading books. The best place in all the world is the library. I absolutely love it.’
I felt the public deserved to know what had been done in these trials. This was probably the most widely reported clinical trial of the past few years, it was a matter of great public interest, and the experiments were performed on children by public servants. So I made a Freedom of Information Act request for the information you would need to know about a trial: what was done, who were the children, what measurements were taken, and so on. Everything, in fact, from the standardised and very complete ‘CONSORT’ guidelines which describe best practice in writing up trial results. Durham Council refused, on grounds of cost.
So I got readers of the column to ask for little bits of information, so that none of us was asking for anything very expensive. We were accused of running a ‘vexatious’ ‘campaign’ of ‘harassment’. The head of the council complained about me to the
Finally, in February 2008, after a disappointing fall in the rate of improvement in GCSE results, the council announced that there had never been any intention of measuring exam performance. This surprised even me. To be scrupulously precise, what they said, in answer to a written question from an indignant retired headmaster, was this: ‘As we have said previously it was never intended, and the County Council never suggested, that it would use this initiative to draw conclusions about the effectiveness or otherwise of using Fish Oil to boost exam results.’
To say that this contradicts their earlier claims would be something of an understatement. In a
Durham county council’s own press release from the beginning of the ‘trial’ reads: ‘Education chiefs in County Durham are to mount a unique back-to-school initiative today which they believe could result in record GCSE pass levels next summer.’ It reports that children are being given pills ‘to see whether the proven benefits it has already brought children and young people in earlier trials can boost exam performances too’. The council’s Chief Schools Inspector is ‘convinced’ that these pills ‘could have a direct impact on their GCSE results … the county-wide trial will continue until the pupils complete their GCSE examinations next June, and the first test of the supplement’s effectiveness will be when they sit their “mock” exams this December.’ ‘We are able to track pupils’ progress and we can measure whether their attainments are better than their predicted scores,’ says Dave Ford in the press release for the trial which, we are now told, was not a trial, and was never intended to collect any data on exam results. It was with some astonishment that I also noticed that they had changed their original press release on the Durham website, and removed the word ‘trial’.
Why is all this important? Well, firstly, as I have said, this was the most well-reported trial of that year, and the fact that it was such a foolish exercise could only undermine the public’s understanding of the very nature of evidence and research. When people realise that they are flawed by design, then exercises like this undermine the public’s faith in research: this can only undermine willingness to participate in research, of course, and recruiting participants into trials is difficult enough at the best of times.
More than that, there are also some very important ethical issues here. People volunteer their bodies – and their children’s bodies – to participate in trials, on the understanding that the results will be used to improve medical and scientific knowledge. They expect that research performed on them will be properly conducted, informative by design, and that the results will be published in full, for all to see.
I have seen the parent-information leaflets that were distributed for the Durham project, and they are entirely unambiguous in promoting the exercise as a scientific research project. The word ‘study’ is used seventeen times in one of these leaflets, although there is little chance that the ‘study’ (or ‘trial’, or ‘initiative’) can produce any useful data, for the reasons we have seen, and in any case it has now been announced that the effect on GCSE results will not be published.
For these reasons the trial was, in my opinion, unethical. You will have your own view, but it is very hard to understand what justification there can be for withholding the results of this ‘trial’ now that it has concluded. Educationalists, academic researchers, teachers, parents and the public should be permitted to review the methods and results, and draw their own conclusions on its significance, however weak the design was. In fact, this is the exact same situation as the data on antidepressants’ efficacy being withheld by the drug companies, and a further illustration of the similarities between these pill industries, despite the food supplement pill industry’s best efforts to present itself as somehow ‘alternative’.
The power is in the pill?
We should be clear that I’m not – and I’m quite entitled to say this – myself very interested in whether fish- oil capsules improve children’s IQ, and I say this for a number of reasons. Firstly, I’m not a consumer journalist, or a lifestyle guru, and despite the infinitely superior financial rewards, I am absolutely very much not in the business of ‘giving readers health advice’ (to be honest, I’d rather have spiders lay eggs in my skin). But also, if you think about it rationally, any benefit of fish oil for school performance will probably not be all that dramatic. We do not have an epidemic of thick vegetarians, for example, and humans have shown themselves to be as versatile as their diets are diverse, from Alaska to the Sinai desert.
But more than anything, at the risk of sounding like the most boring man you know, again: I wouldn’t start with molecules, or pills, as a solution to these kinds of problems. I can’t help noticing that the capsules Durham is promoting cost 80p per child per day, while it spends only 65p per child per day on school meals, so you might start there. Or you might restrict junk-food advertising to children, as the government has recently done. You might look at education and awareness about food and diet, as Jamie Oliver recently did very well, without recourse to dodgy pseudoscience or miracle pills.
You might even step away from obsessing over food – just for once – and look at parenting skills, teacher recruitment and retention, or social exclusion, or classroom size, or social inequality and the widening income gap. Or parenting programmes, as we said right at the beginning. But the media don’t want stories like that. ‘Pill solves complex social problem’ feels much more like a news story than anything involving a boring parenting programme.
This is partly due to journalists’ own sense of news value, but it’s also a question of how stories are pushed.
