Dr. Oz tries to do science again, this time with green coffee bean extract

Dr. Mehmet Oz hosts a popular TV show in which he promotes all sorts of medical treatments, some good and some - well, not so good.  And once in a while, he tries to do a science experiment, as he did in 2011 with a badly flawed experiment on arsenic in apple juice.

Well, Dr. Oz has done it again.  This time he wanted to re-examine a claim that he himself had made on an earlier show about green coffee bean extract.

In April 2012, Oz aired a segment on his TV show called "Green Coffee Bean Extract: The Fat Burner That Works!"  On it he "this miracle pill can burn fat fast, for anyone who wants to lose weight." Not surprisingly, sales of green coffee bean extract skyrocketed in response.

"A marketing apocalypse was ignited!" Dr. Oz pointed out in his show in September of 2012.  "I was surprised by the firestorm," he said.

Dr. Oz loves this topic, by the way. He's run dozens of shows on weight-loss gimmicks, such as "The New Silver Bullet for Weight Loss" in which he promoted a new diet pill called Qnexa, and "Ancient Ayurvedic Secrets to Lose Weight".  But let's leave those for another day.

One problem with Oz's first green coffee bean show was that he based it on a study that has some serious problems.  That study claimed that a particular brand of green coffee bean extract called GCA led to significant weight loss.  Subjects lost a lot of weight, too: 8 kilograms (over 17 pounds) on average.    Dr. Oz called it "a staggering, newly released study."  Wow, must be good, right?

Let's look at that study, shall we?  First, it only involved 16 people, a tiny sample.  There were 3 treatments: high dose GCA, low dose GCA, and placebo.  The subjects were divided into 3 even smaller groups, but not by treatment: instead, each group took all 3 treatments, for 6 weeks at a time, with a 2-week rest period in between.  The only difference between groups was the order of the treatments (high-dose/low-dose/placebo).  Subjects in all 3 groups lost about the same amount of weight.  What was the difference?  Well, the authors claimed that the amount of weight loss during the periods when the subjects were taking GCA was greater than when they weren't, even though they lost weight even during placebo treatment.

One critique of the study is there was no proper placebo control.  Looking at the paper, it is impossible to tell how much weight loss is being attributed to the green coffee beans rather than the daily monitoring of diet, which is known to help with weight loss.  And it's a really, really small study.

Perhaps a larger problem is that the trial was carried out in India, and then written up by a U.S. researcher, Joe Vinson from the University of Scranton, as revealed by a story in The Globe and Mail (Canada) last December.  That's right: the subjects were recruited in India, all data was collected there, and the data was emailed to Vinson so he could write it up.

Even more troubling was that Vinson was paid by the makers of GCA to write the study.  Worse yet, the paper states that "The authors report no conflicts of interest in this work." When asked about this by The Globe and Mail:

"Vinson said that he doesn’t gain financially if the company sells a lot of product and that the journal didn’t require him to disclose the relationship."

This small, badly run study was anything but "staggering", as Dr. Oz called it.  I have little confidence that the data sent to Vinson from India was even correct.

Maybe Dr. Oz might was worried too, because a few months after his original show, he ran another show in which he looked at green coffee bean extract again.  He said he was responding to criticism of his earlier show, and he wanted to set the record straight.  For his second show, "Green Coffee Bean Extract: The Answer to Weight Loss?" he ran his own experiment:

"For the first time, we are doing an unprecedented experiment," he said. "We're doing our own study, right here on this show.... the first of its kind EVER on television!"  

Oz's experiment involved 100 women - all of them in the studio audience for his show - who took either green coffee bean extract or a placebo pill for two weeks.  And the result?  I won't make you watch the video; here is the entire statement of results, from Oz's website:

"In two weeks, the group of women who took the green coffee bean extract lost, on average, two pounds. However, the group of women who took the placebo lost an average of one pound – possibly because they were more aware of their diet for that two weeks because of the required food journal."

On the show, Oz stated proudly: "green coffee bean worked for us."

Maybe Dr. Oz's science experiment was better than the Vinson study.  But that doesn't mean it was any good.  First off, Oz seems to have ignored some critical rules on how to run a experiment involving humans.  As Scott Gavura pointed out at the Science-Based Medicine blog, Oz's study "makes a mockery of good research methodology."  Oz failed to explain how the women were recruited for the experiment, and Gavura points out that apparently did not obtain the ethics board approval that all experiments on human subjects require.

Oz also seems willfully ignorant of the notion that 2 weeks is far too short a time to assess the value of a weight-loss treatment.  Will he go back to those same women a few months later to see if the effect lasted? Somehow I doubt it.

But what about that result?  The women who took the coffee bean extract lost 2 pounds, versus just 1 pound for the other group.  (Actually, thanks to Scott Gavura, we know that the difference was even smaller, just 0.76 pounds.) Oz provides no statistical analysis to demonstrate that is even marginally significant. Nor does he provide the raw data that would allow others to replicate his analysis, as he might have to do if he were actually to try to publish his study. But for Oz, what he described on his show seems to be proof enough.  That's is a poor excuse for science.

Meanwhile, sales of green coffee bean extract continue to climb.  My advice: save your money.  And the next time Dr. Oz runs a science experiment, be skeptical.