Over-hyped claims that antidepressants cause autism and antidepressants

Over-hyped, overstated, and probably just wrong.

That's my summary of the latest high-profile study of autism, which reports that mothers who take antidepressants increase the risk of autism in their unborn children by up to 87%. The new study, which appeared this week in the journal JAMA Pediatrics, received widespread attention, both uncritical (Washington Post, Huffington Post) and more cautious (CBS News). But it was that 87% increase that caught most people's attention.

Many scientists, including me, read this news with skepticism. It seems particularly unlikely given that exactly two years ago, another large study reported exactly the opposite conclusion. The 2013 study, published in the New England Journal of Medicine, found that antidepressant use during pregnancy was NOT associated with an increased risk of autism. What's more, the 2013 study looked at exactly the same class of antidepressants, selective serotonin uptake inhibitors (SSRIs), as the new study.

So what's going on? Was the 2013 study just wrong? It seemed the only way to answer this was to read the new study, written by Anick Bérard and colleagues. Looking over the new numbers, my conclusion is that Bérard simply tortured the data until she got the results–and the press headlines–that they wanted. Let's look a bit more closely.

Bérard and colleagues looked at 145,456 children born in Quebec between 1998 and 2009. From this total, 4,724 were born to mothers who took antidepressants (SSRIs) at some time during their pregnancy. The number of children diagnosed with autism was 1,054, about 0.7% of all babies. Only 46 of the 1,054 were born to mothers who had taken antidepressants. The study's main results concerns mothers who took antidepressants in the second or third trimester: these women accounted for 2,532 infants, of whom 31 were diagnosed with autism.

If these numbers seem confusing, try focusing on just one number: 31. The study's main conclusion–and all the headlines–are based on those 31 children diagnosed with autism whose mothers took antidepressants in their second or third trimesters.

The key results are found in Table 2 of the study. Out of 9207 infants whose mothers took antidepressants one year before getting pregnant, 82 were later diagnosed with autism. Bérard et al. found that, after adjusting for various confounders, this group of infants had no increased risk of autism. Another group was infants whose mothers took antidepressants, in the first trimester: of these 4200 infants, 40 were diagnosed with autism. Bérard et al. computed that the adjust risk for this group was 16% lower than average. This difference was not statistically significant, though.

Finally, there's the third group of 2,532 children whose mothers took antidepressants during the 2nd or 3rd trimesters. 31 of these children were later diagnosed with autism, which worked out to an increase in relative risk of 87%.

The first thing to note here is that the increase is relative, not absolute. The overall risk of autism in this study, which was consistent with other studies, was 0.7%. An 87% increase works out to a risk of 1.3% – that's an increase of 0.6% in the rate of autism. This doesn't sound nearly so dramatic as 87%.

But keep in mind that this supposed increased risk of autism is based on just 31 cases. Digging a bit deeper, we find that the way these 31 children were diagnosed was not so clear: the authors wrote that

"Autism spectrum disorder was defined as a medical service claim or hospitalization with a diagnosis of ASD."

Apparently this means only that the children were evaluated for autism, as Dr. Alison Stuebe at The Huffington Post pointed out. Bérard et al. admit this in their paper, where they report that when they restricted their analysis to children whose diagnoses were confirmed by a psychiatrist or neurologist, the number of children with autism was smaller and the increased risk was not statistically significant. In other words, if they looked only at children with confirmed autism, their main conclusion would fade away.

You also might have noticed that Bérard divided the data up in multiple ways to look for an increase in autism: they looked at mothers who took antidepressants before getting pregnant, during the first trimester, and during the 2nd and 3rd trimesters. They don't report any findings for the 2nd trimester alone, or for the 3rd trimester alone, which would have certainly involved smaller (and probably less significant) numbers.

This raises a potentially fatal problem with the study: multiple testing. Whenever a study considers more than one hypothesis, the statistics must be adjusted to account for that. If you look for an effect in ten different ways, you're more likely to find something by chance alone, so you have to find a much stronger effect in order for it to be valid. (The web comic xkcd has a great explanation of this.) Bérard doesn't explain how many hypotheses she tested, but she does write that:

"No adjustment was made for multiple comparisons; hence, we cannot rule out chance findings given the number of comparisons made."

No kidding! Upon reading this, my main question was how the journal editors let them get away with this. I guess JAMA's editors like headlines, perhaps a bit too much.

The senior author of the study, Anick Bérard, appears to have an agenda, as Slate's David Auerbach explained. Last year, she testified in a lawsuit against Pfizer, claiming that their antidepressant Zoloft caused birth defects. The judge in that case, Cynthia Rufe, threw out Bérard's testimony with the explanation that her methods were unscientific:

"Dr. Bérard’s opinions regarding Zoloft are only made possible by her departure from use of well-established epidemiological methods." 

Now, the fact that Bérard has previously testified in court cases doesn't prove that her current study is flawed, but it does indicate that she has a bias against antidepressants. This bias might explain why her study looked so hard to find an effect when the data don't seem to support it.

Finally, I should note that even if the new study is correct (and I doubt it), it completely ignores the risk of stopping medication for pregnant women with severe depression, as Dr. Alison Stuebe discussed at length at the Huffington Post. (Recall that the 2013 study I mentioned above found that there was no increased risk of autism in women taking SSRIs.) No woman should go off her medication based on this study, although I fear that the headlines from last week will have exactly that effect.

Cashing in on fears of autism

Autism has been the subject of much controversy in recent years.  Rates of diagnosis are rising, and many parents-to-be are understandably worried.  Despite the efforts of many good scientists, we still don't have an explanation for most cases, although we do know that vaccines have nothing to do with it. Many unscientific claims are out there, some of them designed to take advantage of vulnerable parents who are desperate for answers. In this climate, any new claim to have found "the" cause of autism ought to be stated very, very carefully, and backed up by solid evidence.

Just recently, scientists at the UC Davis Mind Institute published a study that described 6 antibodies found in the blood of some mothers of autistic children, and much less often found in mothers of normally developing children.  They suggested that these antibodies somehow got into the brains of developing fetuses, causing autism in the children.  They even gave a name to this form of autism: maternal autoantibody-related, or MAR autism.

If true, this study suggests that a test for these antibodies might predict whether or not a child will have autism.

The study seems plausible, and it was published in a respectable journal called Translational Psychiatry.  Unfortunately, though, the study and the way it has been promoted are plagued with problems.

Perhaps the biggest red flag is that the two lead authors, Daniel Braunschweig and Judy Van de Water already have a patent on the proteins described in their paper, and Van de Water is involved with a company, Pediatric Biosciences (PBI), that is already marketing a test to predict autism based on this study.  Van de Water is the Chief Scientific Advisor for the company, which has licensed her patent for this specific test.  The company website claims

"To date, the test has demonstrated 100% accuracy - meaning if a mother or prospective mother has developed the antibodies, then her child will later be diagnosed with AU [autism] or ASD [autism spectrum disorder]."

Wow. This sounds incredibly accurate. This test is apparently the sole product of PBI, and they trumpeted the new study with a press release claiming that

"PBI is developing a diagnostic test based on these findings that will provide physicians with a reliable set of biomarkers for pre-conception … diagnosis of this Maternal Autoantibody-Related (MAR) form of autism."  

PBI is marketing the test to all "women over 30 who are at least 2 times more likely to give birth to an autistic child" as well as other women.

So, with all this hype, and despite the major conflicts of interest of the lead authors, did the study show what they are claiming?  Is this a test that mothers should take?

In a word, no.

First, the study doesn't even answer the right question.  The study looked at antibodies found in the blood of mothers some years after their pregnancy, not during or before pregnancy. Therefore it doesn't tell us whether a test of a pregnant mother will predict anything at all.  Post hoc analyses like this are notoriously inaccurate: for example, it's easy to find factors that predict every presidential election for the past 50 years, but many of them will perform poorly at predicting future elections.  Similarly, finding 6 proteins that distinguish some mothers of autistic children from mothers of normal children, all measured after the fact, tells us nothing about whether we can predict autism in advance.

Second, their striking claim of 100% accuracy is false.  (Their press release only claimed 99%: "A positive result would mean that they have a 99 percent likelihood of having a child with autism if they proceed with a pregnancy.")  But the study provides no such evidence.  What it did show was that 56 mothers (out of 246) had antigens to one of 12 combinations of 3 antigens that they selected.  [Aside: there are many other combinations of antigens that they did not report, and no indication that they properly adjusted their statistics to account for this.] But the study failed to say whether these 56 mothers also had non-autistic children.  If they did, then even the mothers in their own study had a much lower chance than 99% of having an autistic child.  Given that many families with autistic children also have other, non-autistic children, touting an accuracy number like this, without proof, is outrageous.

The study isn't completely worthless: it points to a possible link between antibodies in the blood of the mother and autism in a child.  If Van de Water were truly interested in the welfare of autistic families, she would be focusing on the obvious next step: test a large, randomly sampled collection of pregnant women for the same antibodies.  Then follow them up, over the next few years, to see if the test can predict the likelihood of having an autistic child.  Meanwhile, no one should be selling a test before studies can show that it has some predictive value.

But Van de Water and her friends at Pediatric Biosciences seem far more interested in making money off the fears of prospective parents.  PBI is planning to charge about $800 for their test, which they'll begin selling next year. Stanford University biostatistician Steven Goodman says that they are "peddling false hope that giving birth to autistic kids can be avoided."

A news article in Science last week raised many serious questions about Van de Water's study and its claims.  Several scientists quoted in that article questioned the premise of the study: that antibodies from the mother somehow get into the brain of the developing fetus. Yale scientist George Anderson said the data are too preliminary and the statistics too weak to support a clinical test (essentially the same argument I'm making here).

And what about that 99% claim?  In their interview with Science, Van de Water and Jan D'Alvise (president of PBI) said it

"was not meant to convey likelihood in the statistical sense, but rather the 99% accuracy with which the study demonstrated specificity of the biomarkers for ASD."  

This is unadulterated poppycock. As of this writing, the company website still claims that the test has "100% accuracy--meaning if a mother or prospective mother has developed the antibodies, then her child will later be diagnosed with AU or ASD [autism]."  That claim couldn't be any more clear - or any more wrong.

With all the controversy over the causes of autism, and with the medical community still struggling to correct the tremendous damage caused by Andrew Wakefield's fraudulent study linking vaccines to autism, the last thing we need is an erroneous claim that someone has found the cause of autism.  And it's far too soon to start offering moms a test that tells them they're going to have an autistic child.