What is Wrong with Testimonials and Results?
October 20 2010
Every trainer and fitness website has a testimonial page with a lot of impressive before and after pictures. And everyone claims it’s all about results. But what is blatantly wrong with these testimonials and results?
And this is the typical strength coach and trainer’s classic line: “I get results and results are all that matters”. Do you?
What is missing in testimonial and results?
Negative results: Have you ever seen a testimonial page with people looking the same in before -after pictures? Or anyone writing about how the program or the supplement didn’t do anything good for them?
For example, the testimonial page shows 10 people who got great results with the diet/workout program. But here might be 9 more people for whom the diet/workout didn’t do anything and 2 more people where the program/diet actually made them worse. These people are never seen of the testimonial page. Do you think the diet/ program still works?
Drop Outs: There are folks who dropped out from a diet or workout program and you will never find their names on a testimonial page. These are the people who usually think the program and supplement didn’t work for them. In drug trials of fatal diseases, like cancer, they usually say “dead men say no tales”.
What about research?
Reporting Bias/Outcome bias: In research this type of under-reporting is called publication bias or/and outcome bias.
In research, you have to report negative results, adverse outcomes, and the study drop outs /attrition rate. The effectiveness of a treatment/supplement is identified by taking into account BOTH the positive and the negative results. If you do not report negative results, you are in violation of the standard code of scientific conduct & ethical behavior and may end up in some legal trouble.
Practical Application
- This is the problem with results/ testimonials. Everyone seem to have results and it is hard to pick one workout/diet over the other based on it.
- Next time when someone talks about results/testimonials, ask them if they ever heard about reporting/outcome bias and send this article to them.
- That’s one more reason to stick with scientific studies.
Related Articles
| Wed October 20, 2010
I’m a big fan of the idea that most diets work provided that they provide a caloric deficit. The best diet is one that you can adhere to for life.
In that regard, I think that good testimonials only reflect those who found the correct diet/training program for them at a time when they were ready for change.
In my practice I use a readiness for change model and try to encourage those who are not in the contemplation, preparation, or action phases not to start a plan until they are truly ready.
Then I believe it is a matter of fitting the right program (both nutrition and training) to the individual and not the other way around.
At the end of the day, this should yield more positive testimonials than not and I have a book full of them.
However, I don’t feel that having failures (we know that 95% of people regain weight) is indicative of failure of a program, but failure of the person to be ready to step up and address barriers a the specific time.
And it would be pretty stupid marketing to advertise failures. 😊
Anoop | Thu October 21, 2010
Hi Mark,
Thanks for the comment.
I am a big fan of the idea that most diet works as long as it is hypocaloric. That’s because it is biologically plauusbile and shown to be true in number of controlled studies. It is not because I saw a bunch of testimonials.
I think the failure of the program or the person really depends on the diet in question. If you put a vegetarian on an Atkins diet or put people on the HGH diet,the person can’t step up much you know. But I see your point.
And the article is not just intended to be about diet. It could be supplements, drugs, therapy, workouts and so on.
| Thu October 21, 2010
Hi Anoop 😊
Great point! I think girls especially fall for inspirational stories and pictures. Seeing a girl’s before and after picture where they look forty pounds lighter in the after picture even gets me thinking, “maybe I should try that diet…”
That’s an eye-opening thing for girls to read and learn about.
Anoop | Sat October 23, 2010
Thanks Sarah for the comment.
And sometimes it is not about marketing. People just really believe in their treatments/workout/products that they are just blind to the negative results. It’s called confirmation bias.
The classic example of the confirmation bias is the lobotomist guy: Walter Freeman who invented the 10- min lobotomy - poking your brain with an ice pick-like instrument through the orbit of the eye, pulling it back and forth to treat mental disroders. It was later labeled one of the most barbaric medical mistakes of modern medicine.
In his final years, after being proven wrong, he sold his house and drove around the country in a camper looking for patients to prove his treatment indeed saved lives of thousands!! That’s the height of confirmation bias.
Sure it worked on a few, but the majority was crippled mentally and those people obviously didn’t send thank you letters to him. He was only recording the the postive hits and ignoring the neagtive hits.
| Sun October 24, 2010
Anoop,
great article as usual.
In the context of the subject, don’t forget about shills. they run rampant in the nutrition and fitness world. Making testimonials even more unreliable.
-Andrew
Anoop | Tue October 26, 2010
Hi Andrew,
Thanks for the comment!
Great point and very true. There is no more con men in the history of any profession than the fitness/Nutrition field.
If people take out all the tstimonials and the author pic, what will people base their judgement on?
| Wed November 03, 2010
Being in the fitness business. I believe there are two peices to every success/Failer story. 100% from the human attempting to lose weight and a 100% from a great workout program. Since the program will always give 100% the fualt can only be placed on the human. If sticking to a meal plan and fitness program 100% there would be no failers. Giving 100% one day and 95% the next day, altering the meal plan becuase he or she has been working hard and feel they can reward themselfs with a diet soda or snickers bar which will lead to failer. So why would I want to embarrass somebody who didn’t stick to the program 100% by posting his failer on my websit?
Anoop | Fri November 05, 2010
Hi Mtfitness,
Thanks for the feedback.
I don’t think every diet or workout program delivers 100%. Most of the diets and workout programs are crap. So it is not always the dieter. Check this article: http://www.exercisebiology.com/index.php/site/articles/which_is_the_best_weight_loss_diet/
And whole point of the article is to say that the effectiveness of a workout, diet, supplement, or drug is determined by looking at both the negative and positive results and not just the positives. You don’t get the full picture by just at the positive results.
>
