When engaging with such people, our differences usually boil down to this: We insist on scientific evidence in the form of randomised controlled trials. But they find testimonials evidence enough.

It is not unusual to be directed to some emotive YouTube video of miraculous healing, complete with ex-patients and piano-accompaniment, and to be told, "See, it works."

Now, some quacks are just heartless bastards in it for the money. Sometimes though, the person on the other side of the phone sounds painfully sincere. For example, some of the people who believe that the HIVEX electromagnetic box effectively treats HIV clearly believe they are doing good. Even after the Advertising Standards Authority recently ordered them to stop claiming that HIVEX is an effective treatment for HIV, they continue to do so, seemingly in the mistaken belief that their anecdotal evidence is rock-solid.

The logical flaw in testimonials

Unlike many of the infomercials on television, miracle cure testimonies often seem honest and heart-felt. This is surely part of why people find them convincing.

A testimonial might go something like this: "I went for HIVEX treatment, the next day I felt better." Even if not expressly stated, the implication is clearly that the HIVEX treatment caused the improvement.

Here we need to make a crucial distinction. On the one hand, believing someone when they say that they feel better is reasonable. After all, if you want to know how someone feels you ask that person. On the other hand though, when someone says that it is the HIVEX treatment that made them feel better they are not testifying to something they experienced, they are drawing a conclusion which may or may not be correct.

It is very difficult for the person to know what caused him or her to feel better. It could have been the placebo effect. It could have been just the natural up and down of health. It could have been social support. It could have the good meal that person had the previous evening. It could even have been that the person is taking antiretroviral treatment.

The claim that it is because of HIVEX is therefore mere conjecture.

Of course, through advertising and the communities that exist around many fake cures, people are encouraged to see causal links where causal links are unlikely. Add some sciency sounding babble and the desperation for a cure that often accompanies serious disease, and it becomes even more compelling to ascribe the healing to the magic box, pill or tonic.

Bursting the quack bubble

It may feel rude to tell someone who just delivered a heart-felt testimony that you don’t believe that the magic pill they just told you about is the cause of there much improved health. However, just letting it go fosters false hope. Testimonials about unproven treatments like HIVEX or Miracle Mineral Solution lead some to forego proven treatments and to put their health and lives at risk.

Double-blind randomised placebo controlled trials are not important because they are Western or some silly reason like that. They are essential because their very purpose is to help us see past the placebo affect, past the natural and random variations in health, and past the biases of those administering the treatment and the biases of those being treated. In short, they are designed to correct everything that makes testimonials unreliable.

The ASA and testimonials

The Advertising Code is pretty sensible when it comes to testimonials. In Section 2 clause 5.3 it states:

Testimonials should not contain any claims to efficacy which cannot justifiably be attributed to the use of the product, and any specific or measurable results claimed should be fairly presented. Where "before" and "after" claims are made, they should be capable of substantiation, expressed and illustrated in such a way as to permit a fair comparison to be made.

I have previously written about a quack HIV treatment called Revivo Tea. They carried testimonials like this one on their website:

"I am over the moon to inform you how REVIVO has changed my life. I was diagnosed HIV positive June 2008 with CD4 count of 386 & I started using Revivo 2009. November 2010 I went for the PCR and bDNA viral load test and the results were undetectable."

After we lodged a complaint with the ASA, Revivo agreed to remove the testimonials. However, all they did was replace phrases like "CD4 count" with "xxx". We lodged a breech complaint and the ASA ruled in our favour. Given that Revivo Tea had two other recent rulings against them we were given the opportunity to ask for sanctions. In early January the ASA ruled that Revivo advertisements must be submitted for pre-publication advice before they could be published by ASA members.

Here are some questions to keep in mind the next time you are confronted with a testimonial:

• How do you know it is the magic treatment that made the person better and not something else?

• How do you know other people will respond similarly to the magic treatment?

• How do you know that the magic treatment might not make some people sick?

• How do you know the magic treatment might not do damage to the body if taken over an extended period?

• Might the person testifying be tempted or socially pressured into ascribing improvements to the miracle treatment?