Bad Science

know what you find.

By now your head is hurting, because of all those mischievous, confusing homeopaths and their weird, labyrinthine defences: you need a lovely science massage. Why is evidence so complicated? Why do we need all of these clever tricks, these special research paradigms? The answer is simple: the world is much more complicated than simple stories about pills making people get better. We are human, we are irrational, we have foibles, and the power of the mind over the body is greater than anything you have previously imagined.

At proper high doses, Cinchona contains quinine, which can genuinely be used to treat malaria, although most malarial parasites are immune to it now.

For pedants, it’s a 30.89C dilution.

General practitioners sometimes prescribe antibiotics to demanding patients in exasperation, even though they are ineffective in treating a viral cold, but much research suggests that this is counterproductive, even as a time-saver. In one study, prescribing antibiotics rather than giving advice on self-management for sore throat resulted in an increased overall workload through repeat attendance. It was calculated that if a GP prescribed antibiotics for sore throat to one hundred fewer patients each year, thirty-three fewer would believe that antibiotics were effective, twenty-five fewer would intend to consult with the problem in the future, and ten fewer would come back within the next year. If you were an alternative therapist, or a drug salesman, you could turn those figures on their head and look at how to drum up more trade, not less.

So, Pinsent performed a double-blind, placebo-controlled study of fifty-nine people having oral surgery: the group receiving homeopathic arnica experienced significantly less pain than the group getting placebo. What you don’t tend to read in the arnica publicity material is that forty-one subjects dropped out of this study. That makes it a fairly rubbish study. It’s been shown that patients who drop out of studies are less likely to have taken their tablets properly, more likely to have had side-effects, less likely to have got better, and so on. I am not sceptical about this study because it offends my prejudices, but because of the high drop-out rate. The missing patients might have been lost to follow-up because they are dead, for example. Ignoring drop-outs tends to exaggerate the benefits of the treatment being tested, and a high drop-out rate is always a warning sign.

The study by Gibson et al. did not mention randomisation, nor did it deign to mention the dose of the homeopathic remedy, or the frequency with which it was given. It’s not easy to take studies very seriously when they are this thin.

There was a study by Campbell which had thirteen subjects in it (which means a tiny handful of patients in both the homeopathy and the placebo groups): it found that homeopathy performed better than placebo (in this teeny-tiny sample of subjects), but didn’t check whether the results were statistically significant, or merely chance findings.

Lastly, Savage et al. did a study with a mere ten patients, finding that homeopathy was better than placebo; but they too did no statistical analysis of their results.

These are the kinds of papers that homeopaths claim as evidence to support their case, evidence which they claim is deceitfully ignored by the medical profession. All of these studies favoured homeopathy. All deserve to be ignored, for the simple reason that each was not a ‘fair test’ of homeopathy, simply on account of these methodological flaws.

I could go on, through a hundred homeopathy trials, but it’s painful enough already.

The Placebo Effect

For all the dangers of CAM, to me the greatest disappointment is the way it distorts our understanding of our bodies. Just as the Big Bang theory is far more interesting than the creation story in Genesis, so the story that science can tell us about the natural world is far more interesting than any fable about magic pills concocted by an alternative therapist. To redress that balance, I’m offering you a whirlwind tour of one of the most bizarre and enlightening areas of medical research: the relationship between our bodies and our minds, the role of meaning in healing, and in particular the ‘placebo effect’.

Much like quackery, placebos became unfashionable in medicine once the biomedical model started to produce tangible results. An editorial in 1890 sounded its death knell, describing the case of a doctor who had injected his patient with water instead of morphine: she recovered perfectly well, but then discovered the deception, disputed the bill in court, and won. The editorial was a lament, because doctors have known that reassurance and a good bedside manner can be very effective for as long as medicine has existed. ‘Shall [the placebo] never again have an opportunity of exerting its wonderful psychological effects as faithfully as one of its more toxic conveners?’ asked the Medical Press at the time.

Luckily, its use survived. Throughout history, the placebo effect has been particularly well documented in the field of pain, and some of the stories are striking. Henry Beecher, an American anaesthetist, wrote about operating on a soldier with horrific injuries in a World War II field hospital, using salt water because the morphine was all gone, and to his astonishment the patient was fine. Peter Parker, an American missionary, described performing surgery without anaesthesia on a Chinese patient in the mid-nineteenth century: after the operation, she ‘jumped upon the floor’, bowed, and walked out of the room as if nothing had happened.

Theodor Kocher performed 1,600 thyroidectomies without anaesthesia in Berne in the 1890s, and I take my hat off to a man who can do complicated neck operations on conscious patients. Mitchel in the early twentieth century was performing full amputations and mastectomies, entirely without anaesthesia; and surgeons from before the invention of anaesthesia often described how some patients could tolerate knife cutting through muscle, and saw cutting through bone, perfectly awake, and without even clenching their teeth. You might be tougher than you think.

This is an interesting context in which to remember two televised stunts from 2006. The first was a rather melodramatic operation ‘under hypnosis’ on Channel 4: ‘We just want to start the debate on this important medical issue,’ explained the production company Zigzag, known for making shows like Mile High Club and Streak Party. The operation, a trivial hernia repair, was performed with medical drugs but at a reduced dose, and treated as if it was a medical miracle.

The second was in Alternative Medicine: The Evidence, a rather gushing show on BBC2 presented by Kathy Sykes (‘Professor of the Public Understanding of Science’). This series was the subject of a successful complaint at the highest level, on account of it misleading the audience. Viewers believed they had seen a patient having chest surgery with only acupuncture as anaesthesia: in fact this was not the case, and once again the patient had received an array of conventional medications to allow the operation to be performed.

When you consider these misleading episodes alongside the reality – that operations have frequently been performed with no anaesthetics, no placebos, no alternative therapists, no hypnotists and no TV producers – these televised episodes suddenly feel rather less dramatic.

But these are just stories, and the plural of anecdote is not data. Everyone knows about the power of the mind – whether it’s stories of mothers enduring biblical pain to avoid dropping a boiling kettle on their baby, or people lifting cars off their girlfriend like the Incredible Hulk – but devising an experiment that teases the psychological and cultural benefits of a treatment away from the biomedical effects is trickier than you might think. After all, what do you compare a placebo against? Another placebo? Or no treatment at all?

The placebo on trial

In most studies we don’t have a ‘no treatment’ group to compare both the placebo and the drug against, and for a very good ethical reason: if your patients are ill, you shouldn’t be leaving them untreated simply because of your own mawkish interest in the placebo effect. In fact, in most cases today it is considered wrong even to use a placebo in a trial: whenever possible you should compare your new treatment against the best pre-existing, current treatment.

This is not just for ethical reasons (although it is enshrined in the Declaration of Helsinki, the international ethics bible). Placebo-controlled trials are also frowned upon by the evidence-based medicine community, because they know it’s an easy way to cook the books and get easy positive trial data to support your company’s big new investment. In the real world of clinical practice, patients and doctors aren’t so interested in whether a new drug