Astroblog: Homeopathy and the Quality of Evidence

Wednesday, February 24, 2010

Homeopathy and the Quality of Evidence

The news is abuzz with the report of the Science and Technology Select Committee, Evidence Check 2: Homoeopathy (read it here). Isn't it amazing, decades of scientific and clinical research showing homoeopathy doesn't work fail to hit the headlines, but let a parliamentary committee even hint that maybe, just maybe as homoeopathy doesn't work the government should pay for it and the story is all over the airwaves. The non-controversial finding that homeopathy is a fancy kind of placebo even made in onto the ABC news.

Naturally, a spokesperson for the homoepaths was a little annoyed; "they dismissed all kinds of evidence" she said. No, that's wrong. They looked at all forms of evidence, put put more weight on strong evidence rather than weak evidence. But this is typical of homoeopaths ability to ignore evidence they don't like. I'll use an example from the evidence presented to the committee itself.

The report is humongous, and I have not read the main submissions in depth, but one thing caught my eye.

testimony of a homoeopath

Mr Wilson: Arnica, which is for bruising, and is
extremely useful in post-operative care. There was a
major trial done on arnica and, indeed, there is one
that has just been published, the Witt Trial, which
was done by the Charite´ Hospital in Berlin. It was a
large trial—3,700 patients involved—and that has
shown clearly that there is a strong benefit in
homeopathic use to these patients with long-term
chronic conditions. One of the subjects of that trial
was arnica."

Okay... Now, the actual trial is here. Complement Ther Med. 2006 Dec;14(4):237-46. Epub 2006 Oct 13. Homeopathic arnica therapy in patients receiving knee surgery: results of three randomised double-blind trials. They investigated the effectiveness of homeopathic Arnica montana on postoperative swelling and pain after arthroscopy (ART), artificial knee joint implantation (AKJ), and cruciate ligament reconstruction (CLR).

Here is the actual description

From November 1996 to December 1997, a total of 343 patients were included in the three trials: 237 in ART, 35 in AKJ, and 71 in CLR

Note the patient numbers, not 3,700 (and the final results were with fewer subjects 57 in CLR 30 test, 27 control). There were numerous tests, and no evidence of correction for multiple statistical analysis (which means the statistics are very likely to give false positives). In only one operation (the cruciate ligament CLR one) was there any indication of a benefit. This should be a red flag, if Arnica is doing something, it should work on all of these operations (and certainly be picked up in the trial with the largest number of patients), this fact screams "false positive".

The benefit in CLR, far from strong, was a marginal reduction in knee
diameter, but not pain, fluid exudate and number of punctures. The only value
they got marginally significant was neither their primary endpoint (relative knee diameter reduction) or their secondary endpoint (absolute knee diameter reduction), but a value obtained by doing an interesting data manipulation which I can't reproduce (ratio of mean difference and pooled pretreatment SD's, in theory not bad, I can't get their values, but its not clear what actual
values they were using). Worse, this effect was significant only for day 1 after surgery, by day 2 (when knee diameters were still increasing), the effect went away. Again, this is consistent with "false positive".

So, all in all NOT a strong benefit shown as claimed by the pro-homoeopathy person. A marginal result in a small trial where the result could easily be due to chance (and has all the red falgs indication it was just a chance result). Hmm, homoeopathy still wildly exaggerating after all these years. Sorry folks, it's not the parliamentary committee ignoring evidence, it's the homoeopaths.

See also Orac and Ben Goldacre for more info.

Labels: scepticism, Science Blogging