Serious doubts about new study of statins and heart disease

The news this past week was filled with reports of a new study in the New England Journal of Medicine (NEJM), which reported a “dramatic risk reduction” in heart attack risk for men using Crestor, a statin drug made by AstraZeneca. This was all over the news, getting headlines on the Sunday (Nov 9) major TV networks and front page reports Monday in The New York Times, the Washington Post, and other major papers. What was striking about this study was that it claimed that people with normal cholesterol levels could get significant health benefits. If true, this new study implies that millions more people should start taking statins to protect themselves against heart attacks.

Wow. Should we all rush out and get some statins? Should we all buy AstraZeneca stock (which went up 20% on the news of this report)? Both?

Hold on a minute. This new finding is rife with problems, despite the breathless news reporting about it. Actually, there are two studies: one published in NEJM, and the second published in Circulation. I’ve read them both. Study 1, in NEJM, got most of the headlines. Study 2 reports on a new diagnostic test that looks at levels of C-reactive protein (CRP), a marker of inflammation. Study 2 found that use of a test called hsCRP – for high-sensitivity C-reactive protein – improved the predictions of cardiovascular risk in men. In other words, the study said, using this test would let you predict more accurately who’s going to have heart problems. Let’s go over the problems one by one. (This is a long blog post, so if you want to know the REAL problem with the study, scroll down to Problem 5 below.)

Problem 1 (raise an eyebrow): Both studies were funded by AstraZeneca, the drug company that sells Crestor. Obviously, AstraZeneca must be pleased that the results suggest that millions more people – those currently considered at low risk for heart disease – should start taking Crestor. However, the funding was disclosed in the reports, and AstraZeneca did not interfere in the analysis, so the funding source does not invalidate the results – not at all. It just makes me a bit more skeptical.

Robert Bazell, a journalist at NBC, was much more credulous. He reported that the study was “squeaky clean.” Well, it’s awfully nice of Mr. Bazell to give his stamp of approval, but disclosure alone does not mean the study had no bias. We’ll get to that in a minute.

Problem 2 (raise the other eyebrow): Both studies also say that high levels of C-reactive protein (CRP) are linked to heart disease, even in men with normal cholesterol levels. The lead author of both studies was Paul Ridker. Paul Ridker owns the patent on the hsCRP test for CRP. Another consequence of these studies is that millions of people are now likely to get tested for CRP, using Ridker’s test. Clearly, Ridker has an interest in the results coming out the way they did. NBC’s Bazell gives him a pass: “As for Dr. Ridker, he says flat out that the financial interest in the test had no effect on the outcome. I certainly believe that. Dr. Ridker has spent most of career working on c-RP and this study validates all his work.”

So let me get this straight: because Ridker has spent his career working on CRP and this study validates his work, we shouldn’t question it? I don’t think so. What this meant to me was that the parties conducting and funding both studies had a very strong interest in the results coming out the way they did. That doesn't mean the results are wrong - again, it just makes me more skeptical. But that’s why we have placebo-controlled trials: to eliminate the effect of bias. So I read the papers, carefully, to see what the data actually said.

Problem 3: the Circulation study didn’t report separately on the effect of CRP and family history of heart disease. In this study, Ridker and colleagues looked at 10,724 men retrospectively (over a 10-year time period), and used a “traditional” model to predict the risk of heart disease. The traditional model had 5 variables: age, blood pressure, smoking status, total cholesterol, and HDL cholesterol. They then added two more variables to the model: (a) the hsCRP test and (b) family history of a heart attack before the age of 60. The report shows that the new, 7-variable model is somewhat more accurate. The study has several methodological problems that I won’t try to describe here, but the biggest problem is that the fail to report the separate effects of the two new variables. In other words, they report only that both variables should be used to measure risk, which means (of course) that patients should be getting the hsCRP test. But what if the entire effect of the study is due to the family history of heart disease? The study doesn’t say. We simply can't tell if the hsCRP test has added value or not. And the leader of this study - Ridker - has the patent on the hsCRP test.

Problem 4: the NEJM study actually reports a very small benefit. All the glowing press reports emphasized the “44% reduction in risk” in those taking Crestor, making it sound very dramatic, but they neglected to report the absolute risk. What I mean here is that if the risk is very, very small, then a relative reduction of 44% is not so significant. Here are the actual numbers: this was a large study, with 17,802 subjects, 8901 getting Crestor and 8901 getting a placebo. The placebo group suffered 251 “events” (one of five cardiac problems, including heart attack), and the Crestor group had 142 events.

This looks pretty significant – and statistically speaking, it is. But the clinical significance is different: you’d have to treat 95 people for 2 years with statins to prevent 1 heart attack. Is that worth it? And if we put millions of people on statins for the rest of their life, which might indeed prevent some heart attacks, will there be other consequences that we can’t yet foresee?
Having an NNT (number needed to treat) of 95 might not sound so bad, but that’s a very high number. An article in Business Week a few years ago pointed out that such high NNT numbers might just represent statistical noise. That article quoted Dr. Nortin M. Hadler, professor of medicine at the University of North Carolina, who said, "Anything over an NNT of 50 is worse than a lottery ticket; there may be no winners.” The article goes on to point out that “an estimated 10% to 15% of statin users suffer side effects, including muscle pain, cognitive impairments, and sexual dysfunction.” Furthermore, it is highly likely that lifestyle changes – getting more exercise, for example – will have a better NNT than 95. So rather than prescribing Crestor, perhaps physicians should explain the greater benefits – possibly much greater - that patients will have from changes in diet and exercise.

Problem 5: This one is the biggest problem of all. The patients in the NEJM study were randomly divided into two groups, treatment (Crestor) and placebo. Table 1 in the paper describes the groups according to a long list of features, and the groups are virtually indistinguishable for most of these – average age, blood pressure, LDL cholesterol levels, body mass index, etc. However, there are 3 critical variables where the two groups are not identical. Presumably this happened by chance, but when you have such a small effect as the one found in this study, small differences can have huge consequences. Let’s look at these 3 variables and at the number of patients in each group (Crestor vs. placebo) with these factors:

Numbers of subjects in Crestor vs. placebo groups

Treatment group:	Crestor	Placebo
Current smoker	1400	1420
Family history of premature CHD	997	1048
Metabolic syndrome	3652	3723

Notice that the Placebo number is higher in all 3 cases. There were 20 more smokers in the placebo group, 51 more people with a family history of CHD (coronary heart disease), and 71 more with metabolic syndrome. All 3 of these variables are risk factors for heart disease – in fact, 2 of them were used by Ridker in his Circulation paper as part of a test to predict risk!

Even if these differences are accidental, all 3 of them put the placebo group at higher risk of heart disease. We don’t know if these totals represent separate people (one person might be a smoker and have a family history of CHD), but if they were, we have as many as 142 more “at risk” people in the placebo group.

Remember, the total number of excess events in the placebo group was only 109 (252 events versus 141 in the placebo group). How many of those events occurred in people with the 3 “bad” variables above? It is entirely possible that these differences in the two study groups could dramatically reduce – even eliminate – the supposed benefit observed in the study.

So what does this all mean? Well, I am not convinced that Crestor has a clinically significant benefit for patients with normal cholesterol. My guess is that further studies, if done properly, will show that the benefit is smaller than that reported last week, and perhaps the benefit is nonexistent.

Finally, on a lighter note, I was pleased that one of my favorite “fake news” reporters, Stephen Colbert, wasn’t fooled at all – he made fun of the study on his show (The Colbert Report) on Wed 12 November. (Scott Hensley over at The Wall St. Journal blog has a nice post about this.) Colbert reported the study in his “Cheating Death with Dr. Stephen T. Colbert, D.F.A.”:

“This is a great breakthrough in the battle to find things to prescribe to people who don’t need them. True, the drug costs $100 a month, but that is a small price to pay to not have the heart attack that there’s no way of knowing if you would have had it.”

Colbert then showed a video clip of Stanford cardiologist Mark Hlatky. Hlatky was interviewed on PBS, where he said “we need to be cautious before we expand the numbers of patients so drastically.” Colbert responded: “sounds like someone hasn’t gotten enough free Crestor pens.”

I guess I haven’t gotten enough free Crestor pens either.