Over-hyped claims that antidepressants cause autism and antidepressants

Over-hyped, overstated, and probably just wrong.

That's my summary of the latest high-profile study of autism, which reports that mothers who take antidepressants increase the risk of autism in their unborn children by up to 87%. The new study, which appeared this week in the journal JAMA Pediatrics, received widespread attention, both uncritical (Washington Post, Huffington Post) and more cautious (CBS News). But it was that 87% increase that caught most people's attention.

Many scientists, including me, read this news with skepticism. It seems particularly unlikely given that exactly two years ago, another large study reported exactly the opposite conclusion. The 2013 study, published in the New England Journal of Medicine, found that antidepressant use during pregnancy was NOT associated with an increased risk of autism. What's more, the 2013 study looked at exactly the same class of antidepressants, selective serotonin uptake inhibitors (SSRIs), as the new study.

So what's going on? Was the 2013 study just wrong? It seemed the only way to answer this was to read the new study, written by Anick Bérard and colleagues. Looking over the new numbers, my conclusion is that Bérard simply tortured the data until she got the results–and the press headlines–that they wanted. Let's look a bit more closely.

Bérard and colleagues looked at 145,456 children born in Quebec between 1998 and 2009. From this total, 4,724 were born to mothers who took antidepressants (SSRIs) at some time during their pregnancy. The number of children diagnosed with autism was 1,054, about 0.7% of all babies. Only 46 of the 1,054 were born to mothers who had taken antidepressants. The study's main results concerns mothers who took antidepressants in the second or third trimester: these women accounted for 2,532 infants, of whom 31 were diagnosed with autism.

If these numbers seem confusing, try focusing on just one number: 31. The study's main conclusion–and all the headlines–are based on those 31 children diagnosed with autism whose mothers took antidepressants in their second or third trimesters.

The key results are found in Table 2 of the study. Out of 9207 infants whose mothers took antidepressants one year before getting pregnant, 82 were later diagnosed with autism. Bérard et al. found that, after adjusting for various confounders, this group of infants had no increased risk of autism. Another group was infants whose mothers took antidepressants, in the first trimester: of these 4200 infants, 40 were diagnosed with autism. Bérard et al. computed that the adjust risk for this group was 16% lower than average. This difference was not statistically significant, though.

Finally, there's the third group of 2,532 children whose mothers took antidepressants during the 2nd or 3rd trimesters. 31 of these children were later diagnosed with autism, which worked out to an increase in relative risk of 87%.

The first thing to note here is that the increase is relative, not absolute. The overall risk of autism in this study, which was consistent with other studies, was 0.7%. An 87% increase works out to a risk of 1.3% – that's an increase of 0.6% in the rate of autism. This doesn't sound nearly so dramatic as 87%.

But keep in mind that this supposed increased risk of autism is based on just 31 cases. Digging a bit deeper, we find that the way these 31 children were diagnosed was not so clear: the authors wrote that

"Autism spectrum disorder was defined as a medical service claim or hospitalization with a diagnosis of ASD."

Apparently this means only that the children were evaluated for autism, as Dr. Alison Stuebe at The Huffington Post pointed out. Bérard et al. admit this in their paper, where they report that when they restricted their analysis to children whose diagnoses were confirmed by a psychiatrist or neurologist, the number of children with autism was smaller and the increased risk was not statistically significant. In other words, if they looked only at children with confirmed autism, their main conclusion would fade away.

You also might have noticed that Bérard divided the data up in multiple ways to look for an increase in autism: they looked at mothers who took antidepressants before getting pregnant, during the first trimester, and during the 2nd and 3rd trimesters. They don't report any findings for the 2nd trimester alone, or for the 3rd trimester alone, which would have certainly involved smaller (and probably less significant) numbers.

This raises a potentially fatal problem with the study: multiple testing. Whenever a study considers more than one hypothesis, the statistics must be adjusted to account for that. If you look for an effect in ten different ways, you're more likely to find something by chance alone, so you have to find a much stronger effect in order for it to be valid. (The web comic xkcd has a great explanation of this.) Bérard doesn't explain how many hypotheses she tested, but she does write that:

"No adjustment was made for multiple comparisons; hence, we cannot rule out chance findings given the number of comparisons made."

No kidding! Upon reading this, my main question was how the journal editors let them get away with this. I guess JAMA's editors like headlines, perhaps a bit too much.

The senior author of the study, Anick Bérard, appears to have an agenda, as Slate's David Auerbach explained. Last year, she testified in a lawsuit against Pfizer, claiming that their antidepressant Zoloft caused birth defects. The judge in that case, Cynthia Rufe, threw out Bérard's testimony with the explanation that her methods were unscientific:

"Dr. Bérard’s opinions regarding Zoloft are only made possible by her departure from use of well-established epidemiological methods." 

Now, the fact that Bérard has previously testified in court cases doesn't prove that her current study is flawed, but it does indicate that she has a bias against antidepressants. This bias might explain why her study looked so hard to find an effect when the data don't seem to support it.

Finally, I should note that even if the new study is correct (and I doubt it), it completely ignores the risk of stopping medication for pregnant women with severe depression, as Dr. Alison Stuebe discussed at length at the Huffington Post. (Recall that the 2013 study I mentioned above found that there was no increased risk of autism in women taking SSRIs.) No woman should go off her medication based on this study, although I fear that the headlines from last week will have exactly that effect.