Dr. Oz tries to do science

TV doc Mehmet Oz, who hosts a very popular daily television show, tried to conduct a scientific experiment this week. He wanted to test whether apple juice from some of the most popular juice makers in the U.S. contained toxic levels of arsenic. For some reason - perhaps he was feeling guilty about all the unscientific claims he has been pushing on his show - he decided he would try to prove this scientifically.

With all the resources of his TV network behind him, he was able to put on a pretty good show. But Dr. Oz is no scientist, as he made very clear when he presented his "discoveries." His pretend-science act - and that's what it was - was a disaster. I'd have to give him an "F" for his experiment, where he claimed that he had found dangerous levels of arsenic in five popular brands of apple juice: Minute Maid, Apple & Eve, Motts, Gerbers, and Juicy Juice.

His website proudly displays these results, claiming that the arsenic is from pesticides used in China. After his "extensive national investigation" he concluded:

"American apple juice is made from apple concentrate, 60% of which is imported from China. Other countries may use pesticides that contain arsenic, a heavy metal known to cause cancer. After testing dozens of samples from three different cities in America, Dr. Oz discovered that some of the nation's best known brands of apple juice contain arsenic."

There are many problems with this study, but the biggest one is that its conclusions are false. The reasons include:

1. Oz asked a lab to test for total arsenic, but there are two forms of arsenic, commonly known as organic and inorganic. Organic arsenic is everywhere, and it's mostly harmless. As the FDA explained in its first warning letter to Oz:

"As we have previously advised you, the results from total arsenic tests CANNOT be used to determine whether a food is unsafe because of its arsenic content. We have explained to you that arsenic occurs naturally in many foods in both inorganic and organic forms and that only the inorganic forms of arsenic are toxic, depending on the amount. We have advised you that the test for total arsenic DOES NOT distinguish inorganic arsenic from organic arsenic."

2. Oz didn't try to replicate his own results, and didn't even think to question the numbers he got from the lab he hired. Here's a hint, Dr. Oz: if you come up with a surprising result, then you must try to replicate it yourself to see if you get the same answer. And you should be your own greatest skeptic. Not Dr. Oz: he had all his samples tested at a single lab, which the FDA warned him was giving erroneously high results. The FDA re-tested samples from the same lot for one of the juice makers (Gerber), and sent Oz another letter on Sept 13, telling him the results, as follows:

FDA Sample Total Arsenic Result (Average)
Sample 659595 Lot #1157515791 2.0 ppb
Sample 659596 Lot #1125515762 4.0 ppb
Sample 659597 Lot #1125515761 6.0 ppb
Sample 710623 Lot #1059515761 5.0 ppb
Sample 710624 Lot #1059515762 5.0 ppb
Sample 710625 Lot #1157515761 2.0 ppb
Sample 710626 Lot #1157515762 3.0 ppb

The lab that Oz hired had reported far higher total arsenic levels: 36 ppb. The highest result that the FDA found was six times lower. This would be very, very worrisome to any good scientist. The FDA also warned in this letter that

"it would be irresponsible and misleading for the Dr. Oz Show to suggest that apple juice is unsafe based on tests for total arsenic."

But of course Oz wasn't going to go back and check his results. That would be too scientific! Instead, he aired a show in which he wildly overstated the results and the possible risks. He didn't express even the slightest uncertainty about his findings. He interviewed audience members who dramatically confessed that they hadn't realized they were poisoning their own children. The horror!

To their credit, the mainstream media didn't buy this one. ABC in particular, featured an interview on Good Morning America with both Oz and Dr. Richard Besser, ABC's medical editor, who called Oz's show "extremely irresponsible" to his face. Oz danced around the accusations but never once admitted he might have made a mistake. Besser really lays it on the line in this smackdown (click to watch the video):

Among other things, Besser said that Oz's hyperbolic, poorly-supported claims are like " yelling fire in a movie theater". Bravo for Dr. Besser.

I'm not sure if Mehmet Oz was always the way he is now, but he seems to use his show to sell products (and to sell himself!) rather than to inform the public about medical issues, as one of my fellow Forbes bloggers, Alice Walton, has noted. Dr. Oz has embraced complete quacks, too, notably Joseph Mercola, whom he has had on the show more than once.

Maybe Oz was trying to show his science-y side by doing what he thought was a real study, actually measuring arsenic levels in samples of apple juice. But he botched it so badly that he just demonstrated his own lack of serious science cred.

Dr. Oz ought retract his erroneous claims about arsenic and apologize for his scaremongering and his sloppy science. But he's showing no signs of that: on his website he is sticking by his claims, and he challenges the FDA and the food companies to come on his show. He's also claiming that the source of the arsenic is pesticide sprayed on apples imported from China, although he presents no evidence at all for that. This is not how science is done, Dr. Oz. You made a claim, and others pointed out major problems with it. It's incumbent on you to come up with better evidence. But somehow I suspect that will be too much trouble for you. Real science can be such a pain that way.