These new RNA vaccines are a triumph for science and medicine

This week the FDA approved a second vaccine against SARS-CoV-2, the virus that causes Covid-19. We now have two highly effective vaccines, one from BioNTech and Pfizer, and the other from Moderna. A third vaccine, from Oxford University and AstraZeneca, is very close to approval.

The two new vaccines, both based on RNA, are both remarkably effective. Below I’ll summarize some of the numbers, which have been published for the world to see.

This is a scientific triumph. Less than a year ago, no one outside China even knew this virus existed. The genome of the virus was first released in January, and within a few months scientists had designed the first vaccines. Clinical trials were launched immediately, and larger trials followed, leading us to where we are today: two new vaccines, tested and validated in tens of thousands of people, now being manufactured and shipped to billions.

For anyone who might be skeptical, or who just might want to know more, the test results are being published openly. The New England Journal of Medicine has a dedicated website with dozens of papers and audio summaries, including results from the large-scale (Phase 3) trials of the Pfizer vaccine.

Before getting into the numbers, let’s summarize what these two new vaccines are. (I wrote about this in July, if you want to read my previous explanation.) Both of them are RNA vaccines, which is itself a dramatic breakthrough. RNA vaccines have been discussed for years, but the technology was never employed for human vaccines until now.

Here’s how they work: our immune system (which is super-complicated, as Ed Yong explained in The Atlantic) recognizes microscopic invaders and destroys them. Once you’ve been infected with Covid-19, the immune system swarms over the viral particles and basically learns what they look like. SARS-CoV-2 has a protein all over its surface called “Spike,” so that’s what the immune system recognizes.

Once you’ve fought off the infection, the immune system remembers what Spike looks like. If you’re infected again, it can respond far more quickly, so you won’t get sick. This is what we call acquired immunity.

So for vaccines, the trick is to teach your immune system to recognize Spike. One way to do that is to manufacture lots and lots of the Spike protein, and put that in the vaccine (sort of–I’m greatly oversimplifying here).

But with modern genomics technology, we can use a different approach. Every cell in your body has machinery inside it to translate RNA into proteins. As soon as we had the SARS-CoV-2 genome, back in January, we knew the genetic code for Spike. So rather than make the protein, what if you just made the RNA, which is far easier and faster to manufacture, and injected that into people? Do our own cells then translate the RNA and make the Spike protein?

Well yes, they do. And not only that, but–as the Modern and Pfizer clinical trials have now proven–our immune system recognizes that the Spike protein is foreign (it’s complicated) and launches an attack.

So to make an effective RNA vaccine, you simply have to inject enough RNA so that the immune system responds. That’s what both the Modern and BioNTech/Pfizer vaccines have done.

Now let’s look at the numbers. As reported in NEJM just two weeks ago, the Phase 3 trail for the Pfizer vaccine tested 43,448 volunteers, of whom 21,720 got the vaccine and 21,728 got a placebo. At the time of the report, 162 people who received the placebo had become sick with Covid-19, but only 8 people in the vaccine group got sick. That’s a 95% reduction in illness, a remarkably good result. They also reported that 10 people had “severe” illness, and 9 of those ten were in the placebo group.

How about the Moderna vaccine? This vaccine has almost identical efficacy, published in a preliminary report a few weeks ago as 94.5%. Just a few days ago, an FDA review panel approved the vaccine and confirmed that its efficacy was above 94%. And the Modern vaccine doesn’t need the super-cold freezers that the Pfizer vaccine needs, which makes it easier to distribute.

Both vaccines have minimal side effects in most people, mostly soreness at the injection site, and sometimes headaches or chills, which subside within a day. RNA is quickly degraded in the body, so there’s no reason to expect any lingering side effects from these vaccines.

There’s also growing evidence that immunity lasts for many months, if not years. Another report in NEJM, on the Moderna vaccine, contains some of the latest data, which shows that immunity is still strong after 4 months. Of course, with a brand-new vaccine, we simply have to wait to see if the immunity lasts for years, but all signs are positive right now.

So yes, these are really good vaccines. I will get mine as soon as I can, although I expect I’ll have to wait several months because of short supply.

(The Oxford/AstraZeneca vaccine, a more traditional protein-based vaccine, has also shown positive results, either 62% or 90%, depending on the dosage regimen, but the 90% results are based on fewer cases. Even so, it is clearly effective and it should be approved soon, at least in the UK. So we might soon have 3 vaccines.)

A note to anti-vaxxers: no, you cannot catch Covid-19 from these vaccines. They don’t contain the virus! They only have a fragment of RNA from one protein, and the virus has RNA that encodes 28 other proteins. It’s simply impossible for the virus to self-assemble without the rest of its genome.

But hey, if you don’t want the vaccine, go to the back of the line. Most of the world is desperate for it.

The success of RNA vaccines is a huge win for science, but even more, it’s a huge win for the human population. We’re still many months away from vaccinating the whole world, but with two highly effective vaccines, we can finally have hope to end this pandemic.

New Nature open access policy is little more than a money grab

We scientists love to publish papers, and we get especially excited when our papers appear in “top” journals. The journals know this, and sometimes it seems they just want to see how much they can get scientists to grovel.

That’s what I was thinking a couple of weeks ago, when the publishers of Nature announced that they will charge authors €9,500 ($11,500) to publish a paper as open access, meaning readers can get the paper without a subscription. They called this, without a trace of irony, their “gold open access option.”

$11,500??? Sadly, the Nature publishers were not kidding.

This is outrageous. $11,500 is more than scientists earn in a year in some countries, as Forbes blogger Madhukar Pai pointed out. What’s truly outrageous is that they’re asking for this payment from a community that does all the work for them for free. If Nature is going to treat scientists like suckers, it’s time we stopped playing along.

Let’s back up a minute and look at how the academic publishing system works. (When I explain this to non-scientists, they are often flabbergasted.) Consider: a typical science paper describes experiments that cost tens or hundreds of thousands of dollars, most of which comes from government grants (the most common source of funding) or from private foundations. Scientists write the paper and then submit it to a journal. The journal, in turn, asks other scientists to review the paper, which they do, using their own time and expertise.

All of this–the scientific experiments, the writing, and the reviewing–is done for free, from the journal’s perspective.

Journals then claim copyright on the papers and charge fees to anyone who wants to read them. Not a bad deal for them: virtually all the labor is free. Scientific journals, most of which are owned by a small number of large, for-profit publishers, are very, very profitable.

The whole system, as Berkeley professor Mike Eisen explained in a recent interview in Science, “was built for the printing press.” When journals had to print everything on paper and ship the journals to libraries around the world, it kind of made sense. They were providing a valuable service for science, and it does cost money to print and deliver all those journals.

For over two decades, though, we have been distributing papers electronically, and there’s almost no need for paper copies. One might expect that journals would change their model, but they haven’t. In fact, they’re even more profitable now than they were before the Internet.

Not content with their enormous profits, it now seems that Springer Nature wants to suck even more money out of academic science. It’s true that Nature publishes some highly prestigious scientific journals, but their announcement of this new “gold” open access policy just drips with self-congratulation. “Research published in Nature and the Nature research journals is downloaded ... over 30 times more than papers in a typical journal,” they write. (Who wants to publish in a “typical” journal after reading that?) They also claim to be an “innovator in open access,” which is, frankly, nonsense. Springer and the other for-profit journals have been fighting open access since the mid-2000s, and this latest announcement is yet one more salvo in their battle against it.

(Or maybe Springer thinks that charging $11,500 to make a paper open access is an innovative move? It does take chutzpah, I’ll grant them that.)

The open access movement, which I’ve long been a part of, wants to make all scientific research freely available to anyone, with no costs or delays. As every scientist knows, science only progresses by sharing its discoveries, and barriers such as subscription fees serve only to slow down that progress. Given that most research is paid for by the public, it makes no sense at all to allow for-profit journals to control access. The only reason they still do is because they’ve done so for decades, and it’s hard to change an entrenched system.

Nature’s outrageously high fee also excludes virtually every scientist from low and middle-income countries, as fellow Forbes blogger (and scientist) Madhukar Pai wrote last week.

Rather than a move to support open access, this new fee is little more than a money grab. It’s actually even worse: in addition to the new $11,500 open access fee, Nature also announced an option (they call it a “new OA pilot”) whereby you pay them $2,600 for a preliminary review, and they evaluate your paper for six of their journals. In this option, they might reject your paper outright, and you’re out $2,600 with nothing to show for it. If they think it’s worthy, you pay the remainder of the open access fee later. Gee, this seems like a great idea–paying $2,600 for something that currently is free. Thanks, Nature!

Of course, Nature journals will still allow scientists to publish papers the old-fashioned way, where they don’t pay the €9,500 fee and where the journal then owns the paper. Rather than doing that, or paying the outrageous fee, let’s hope this money grab makes scientists look elsewhere for a place to publish their findings. And while we’re at it, let’s tell the Nature editors we won’t be reviewing for them any longer, not while they’re charging this ridiculous $2,600 fee for a service that we scientists have been providing for free. I’ve already done that once, and I plan to continue until they drop this idea.

A new study out of Denmark tried to measure the benefits of masks. It didn't go so well.

Social media has been abuzz this week over a new study out of Denmark about the effectiveness of masks on the risk of getting Covid-19. Depending on what news source you looked at, you might have heard that masks might not protect the people wearing them, or that wearing masks doesn’t prevent the spread of the virus. You might also have read the near-immediate backlash in which scientists pointed out all the evidence that masks really do work.

I read the study. It doesn’t prove anything, as even its own authors admit.

Let’s dig into the actual results just a bit to see what all the fuss is about.

The study was conducted in Denmark in April and May of this year, and what it tried to do (not very effectively) was to measure the effect of a recommendation to wear masks. That’s right, they weren’t really measuring the benefits of masks directly at all!

The study enrolled about 6000 volunteers, and for half of them they recommended wearing masks whenever they went outdoors. They also provided masks to those volunteers. For the other half, they didn’t do anything. At the time (March and April), Denmark was recommending social distancing, but universal mask wearing wasn’t recommended.

Quite a few people dropped out, so in the end they only had 4862 people in the two groups, about 2400 per group.

What did they measure? Well, they did an antibody test (far from perfect, but let’s not digress) at the beginning of June to see whether or not people were infected with SARS-CoV-2, the virus that causes Covid-19.

(Note that they DID NOT measure how well the masks might have protected anyone else in the community. They were only measuring whether a mask might protect the wearer.)

And the results? 42 people in the mask-recommendation group were infected, and 51 people in the no-recommendation group were infected. (That’s 1.8% versus 2.1% of each group.) So there was a small reduction, but it was not statistically significant, which means we really can’t say if the mask recommendation helped prevent infection.

The study authors admitted this themselves, writing: “the findings are inconclusive ... compatible with a 46% decrease to a 23% increase in infection.” In other words, the results could mean that masks reduce self-infections by 46% or increase them (bizarre as that sounds) by 23%.

In other words, this experiment doesn’t tell us much. If it weren’t about Covid-19, I doubt that the Annals of Internal Medicine would have published it. (A commentary by my colleagues at Hopkins and Stanford suggested that Annals was right to publish it, as long as scientists “carefully highlight the questions that the trial does and does not answer.”)

Now some big caveats. First, in the mask recommendation group, only 46% of the participants wore masks as recommended. Second, the study didn’t ask if anyone in the no-recommendation group wore masks. Third, the study relied on self-reporting to determine who was actually wearing their masks consistently–that is, they simply asked the participants to tell them how often they wore their masks.

What? Imagine if you were studying the use of seat belts to reduce injuries, and only 46% of the people you told to wear seat belts wore them properly. On top of that, imagine that you relied entirely on self-reporting to determine who was actually wearing the seat belts. This study design is nearly worthless if you want to know the true benefits of seat belts.

So the Danish mask study was inconclusive, as its own authors report. Therefore it would be a huge mistake, scientifically speaking, to take this non-result and conclude that masks do not protect you. It would be an even bigger mistake to conclude that the study showed that masks don’t benefit the community. Unfortunately, that didn’t stop these two Oxford University scientists from jumping to exactly that conclusion. They claimed, in an article in The Spectator, that the study showed that “wearing masks in the community does not significantly reduce the rates of infection.” This is dead wrong. The study wasn’t even measuring community rates of infection.

(For an excellent Twitter-take on what the study does not prove, see this thread from The Health Nerd.)

And as the CDC has documented, multiple studies have already shown that masks are highly effective at limiting the spread of Covid-19. And a study last summer pointed out that increasing the use of masks by just 15% “could prevent the need for lockdowns and reduce associated losses of up to $1 trillion” in the U.S. alone. So yes, it’s a good idea to wear a mask.

It’s also simply common sense that if you wear a mask, the amount of virus that you breathe in or out will be reduced. You’re not just protecting yourself, you’re also protecting everyone around you.

Until we can get this pandemic under control, we all need to wear masks in public. It’s utterly ridiculous that this has become a political issue, as it has in the U.S. To those who think mask wearing somehow limits their personal freedom: get over it. When you see a red light at a busy intersection, do you race through it because you need the “freedom” to drive like a crazy person? No. Civilized society requires everyone to follow some basic rules to protect each other, and during a pandemic, wearing a mask is one of them. And to those who think that wearing a mask somehow shows their civic virtue? No, that’s wrong too. To use the same example, stopping at a red light doesn’t prove that you’re virtuous.

It’s just a mask. It’s not a political statement. Get over it.

New Alzheimer's disease fails, then works, then fails again


 Alzheimer’s disease is one of the most devastating conditions of old age. By recent estimates, more than 5 million people in the U.S. have Alzheimer’s, and managing the disease will cost over $300 billion in 2020. As the population ages, this problem is growing worse, and yet we still have no effective treatment.

You might have seen rosy-looking ads for Alzheimer’s treatments, but nothing really works, not yet at least. That’s why many people were excited about the possibilities of a new drug, aducanumab, that showed early signs of being able to reduce the accumulations of “plaques” in the brain.

Background: in people with Alzheimer’s, a protein called beta-amyloid accumulates in the brain, forming plaques that seem to disrupt brain function. (This hypothesis is not fully proven, but it is widely considered credible.) Thus one way that we might treat Alzheimer’s would be to reduce or eliminate beta-amyloid plaques. That’s what Biogen’s new drug, aducanumab, is intended to do.

Biogen has run two separate trials, called “EMERGE” and “ENGAGE,” to test whether or not aducanumab (ADU for short) worked.

Here’s where things get murky. Back in March of 2019, both trials were halted due to “futility,” because ENGAGE was showing no benefits for the new drug. In EMERGE, the high-dose patients seemed to be getting some benefit, but Biogen had specified ahead of time that if either trial was failing, that would mean the drug wasn’t working. Thus they halted the trials, disappointing as it was.

Fast forward to October of 2019, though, and Biogen had a new story. They went back and looked at a subset of the patients in ENGAGE (the study that had failed), and said that there was a benefit after all, if they looked only at the high-dose patients. This past July, Biogen went to the FDA and applied for approval for ADU.

Just this past week, two very conflicting announcements about ADU appeared. First, on Wednesday, the FDA’s internal scientists released a very rosy report, saying that the data from one of the trials were “robust and exceptionally persuasive.” For the second trial, the scientists said that even though the drug initially seemed to fail, a closer review led them to conclude that overall, ADU did provide a benefit.

Biogen shares rose 45% that day, adding $17 billion to the company’s value.

Then on Friday, a panel of independent, external scientists released their conclusions, which resoundingly rejected the drug. The external panel said that the data from the two trials was unconvincing, and they pointed out “multiple red flags” in the analysis.

(Trading in Biogen stock was halted during the Friday meeting, but at the end of the day it was close to the high it reached on Wednesday.)

So what happened? It appears to be a classic case of cherry-picking: when the data from the ENGAGE study didn’t pan out, the company re-analyzed a subset of the data and found a more-positive picture.

That’s not really kosher, as explained by a separate group of scientists in a paper published just a few days ago in the journal Alzheimer’s & Dementia. In this paper, David Knopman and colleagues, from the Mayo Clinic and Stanford Medical School, analyzed data that Biogen has released from its two trials. (The trials haven’t been published, but some of the findings were released in a publicly-available slide presentation that the authors relied upon.)

Knopman and colleagues explained that after two conflicting trials, there simply isn’t enough evidence that ADU works, and they also offer alternative explanations for the positive findings. They argue that the best Biogen can do is to “perform another trial of high‐dose ADU of at least 78‐weeks duration,” which could determine whether or not the positive results were real or just coincidence.

It’s somewhat mysterious that the FDA’s internal panel released their rosy report about ADU on Wednesday, only to be slapped down just two days later by an independent outside panel of scientists. After reading the negative views of the external panel and the analysis in the paper by Knopman and colleagues, I’m very skeptical that ADU has any clinically significant effect. If it had a truly robust effect, it simply wouldn’t be so hard to tease it out.

So we still don’t have a good treatment for Alzheimer’s, but the world still needs one.

The newest member of the coronavirus task force is giving out terrible advice


Why shouldn’t we trust the advice of an M.D. from Stanford University? Because he’s unqualified, that’s why.

As everyone with a pulse knows, the U.S. has handled the coronavirus pandemic very, very badly. Tragically, over 220,000 people have died, and our rate of deaths per capita is higher than any other country in the world.

The Trump administration established a coronavirus task force back in the spring, supposedly led by VP Mike Pence. For a long time, the task force included Dr. Anthony Fauci, a world-renowned expert on viruses who is also the Director of the NIH’s infectious disease institute, where he has worked for 40 years. Despite the frequently erroneous and misleading statements by President Trump, Dr. Fauci consistently gave the public advice that was both scientifically and medically accurate. He never promised that the virus would simply disappear, and he urged everyone to wear masks and avoid unnecessary contact with others. He also warned against re-opening businesses too quickly.

Trump didn’t like that, so he pushed Fauci to the sidelines in favor of someone whose advice matched what he wanted to hear.

Enter Scott Atlas. Atlas is a Fellow at the right-wing Hoover Institute at Stanford University, where he studies health care policy. He’s also an M.D., a radiologist who specializes in MRIs. Notably, he has no special expertise on viruses, vaccines, or epidemiology.

Atlas has pushed for schools to reopen and for college sports to resume, against the advice of public health experts. Just last week, he tweeted that masks don’t work, a claim that was so outrageous and dangerous that Twitter took it down. (To be precise, Atlas’s tweet was “Masks work? NO”.) Another coronavirus task force member, Dr. Deborah Birx, said she felt “relief” that Atlas’s tweet was removed.

Perhaps Atlas is so convinced of his own brilliance–after all, Stanford is one of the world’s top universities, and he wason the faculty there–that he thinks he’s an expert on everything. But a good scientist would pay attention to the recommendations of others who are clearly more qualified, and Atlas has not done that. For example, he has argued, against the evidence of experts, that “low-risk groups getting the infection is not a problem” (wrong–they can spread the infection to high-risk groups) and that only people with symptoms should get tested (very bad idea, given that many people are asymptomatic).

Alarmed at the harm that Atlas’s advice has been causing, a group of more than 70 of his Stanford colleagues, including world-renowned experts in infectious diseases, epidemiology, and health policy, published an open letter decrying Atlas’s bad science. Their statement read, in part:

“... we have both a moral and an ethical responsibility to call attention to the falsehoods and misrepresentations of science recently fostered by Dr. Scott Atlas, a former Stanford Medical School colleague. Many of his opinions and statements run counter to established science and, by doing so, undermine public-health authorities and the credible science that guides effective public health policy.”

Atlas responded by threatening to sue his Stanford colleagues over their letter, and in response to that, an even larger group of Stanford professors released a statement saying they wouldn’t be intimidated. The second letter, with over 100 signatories, stated:

“We believe that his [Atlas’s] statements and the advice he has been giving fosters misunderstandings of established science and risks undermining critical public health efforts.”

My bottom line: Atlas is a bad scientist, apparently far more interested in power and influence than in public health. I’m not commenting on his skills as a radiologist, which are irrelevant here. (He might be an outstanding radiologist.) However, he’s providing advice to the U.S. government that contradicts the advice of scientists who are far more qualified than he is, and when they pointed that out, rather than buttressing his arguments with data, he threatened to sue. That is not the behavior of a good scientist.

To those who think that this disagreement means there are two sides to the issue, I urge you to think again. Science and medicine are highly specialized. Just as you wouldn’t want a virologist to read your MRI scan, you wouldn’t want a radiologist (Atlas) to decide on the best way to treat the Covid-19 virus. So when a radiologist (Atlas) disagrees with a virologist (Fauci) over a virus, guess who’s most likely to be right?

It’s unfortunate that the current administration has chosen to heed the advice of someone who tells them what they want to hear, rather than someone who truly has the qualifications to advise them.

Can the SARS-CoV-2 virus damage the brain?

A certain very famous politician came down with Covid-19 recently, and has been acting even more erratically than usual. This has led a number of pundits (and some doctors) to speculate that this politician’s behavior might be a symptom of his ongoing infection. Could this be true?

Well, maybe. Most of the attention around Covid-19 has been focused on the damage that the SARS-CoV-2 virus causes in the lungs, which can lead to difficulty breathing, the need for a respirator, and even death. The virus has the ability to replicate explosively in a person’s lungs, not only causing serious damage but also triggering an over-reaction by the immune system, a so-called “cytokine storm” that itself can kill you, even if the virus doesn’t.

However, numerous reports have shown that the virus gets into many other tissues besides the lungs, including the brain. Just this week, a new study out of Northwestern University School of Medicine found that over 80% of patients with Covid-19 had at least some neurological symptoms. 80% is a startlingly high number.

While that sounds alarming, let’s look at the details. The new study looked at 509 Covid-19 patients, all of them admitted to hospitals in Chicago. These were “consecutive” patients, meaning that the investigators didn’t cherry-pick their subjects, but just took 509 in a row. That seems sound.

Most of the symptoms, although definitely affecting the brain, were mild. 38% of the symptoms were headaches, and 44% were “myalgias”, which refers to aches and pains throughout the body. (Note that some patients had more than one type of symptom, so the numbers in the study add up to more than 100%.)

However, 32% of the patients had encephalopathy, which can be much more serious than a simple headache. According to NIH, encephalopathy can involve:

“loss of memory and cognitive ability, subtle personality changes, inability to concentrate, lethargy, and progressive loss of consciousness.”

Does this sound like any of the behaviors we’ve seen in our most famous infected politician?

The new study is not the first one to report neurological symptoms caused by Covid-19. Back in July, a research team from University College London reported multiple cases of neurological problems in their cohort of 43 patients. They observed not only encephalopathy (in 10 patients), but also encephalitis in 12 other patients and strokes in 8 more. Some of the patients in that study were reported as experiencing “delirium/psychosis,” and strokes often cause permanent brain damage. Clearly, the SARS-CoV-2 virus can cause serious health problems, and disturbing behavioral changes, if it gets into the brain.

None of this means that any current political leader is experiencing an altered mental state. We don’t have a direct test that measures whether the virus is present in a person’s brain, so all we can do is observe symptoms and make inferences from those. The best available evidence today, though, shows that for anyone with Covid-19, neurological problems are definitely something we should be worried about.

Why do the Covid-19 vaccine trials take so long?

The whole world is waiting for a Covid-19 vaccine. More than 100 different vaccines are being investigated, and 42 of them are already being tested in humans, which is lightning-fast progress in the world of vaccine development.

11 vaccines are already in Phase 3 trials, which use thousands of volunteer subjects to test whether a vaccine really works. If any of these 11 trials are successful, as many scientists expect them to be, then the world might finally begin the process of opening back up.

By all accounts, though, we’re still a few months away from having an approved vaccine. Why does this take so long? Today I’ll try to answer this question. A little math is involved, but we don’t need much to get the basic idea across.

In a Phase 3 trial, we give the vaccine to large numbers of people to see if it works. Some of the 11 current trials use as many as 40,000 volunteers, so let’s use that number for the sake of discussion. In the trial, we might give the real vaccine to half the volunteers–20,000 people–and give a placebo to the other 20,000. A placebo is a harmless shot, typically just saline solution, that won’t have any effect. The volunteers don’t know if they’re getting the real thing; this is called “blinding.”

Then we wait. Here’s the problem: we don’t infect anyone intentionally, so we have to wait for naturally-occurring infections, and it might take a long time to see those. Subjects just go about their lives, and if they get sick, the study records that fact.

So the question is, how many people in each group of 20,000 will be infected in the first week? The first month? Two months? The answer is that we simply don’t know. To speed things along, scientists running the trials try to select volunteers who are more likely than most people to get infected, but we can’t really control the number of people who get sick.

Let’s suppose that after just one week of a trial, 3 people in the placebo group come down with Covid-19, and no one in the vaccine group gets sick. So far so good, right? But we can’t possibly conclude that a vaccine works based on just 3 cases. Statistics tells us that those 3 cases might have just happened by chance. (More precisely, if 3 cases occur in the 40,000 subjects, and if the vaccine doesn’t work at all, then there’s still a 12.5% chance that all 3 cases will occur in the placebo group.)

Suppose that 2 months roll by, and now we have 100 people in the placebo group who got sick, and only 10 infections in the vaccine arm. This is much, much better: without going into the math, a difference of 100 versus 10 would be highly significant, suggesting that the vaccine reduced cases by 90%.

But what if 2 months roll by and the placebo group only has 10 cases? Even if the vaccine group has zero cases, such a small number is not going to be enough to give us confidence that we have an effective vaccine. We want to see as many cases as possible–but we can’t force the issue. We have to wait.

In the US, the FDA has announced that a vaccine has to protect at least 50% of people in order to be declared effective. This means we need to see enough cases in to be confident that a vaccine confers that degree of protection. 50% is a pretty low bar, but so far none of the trials have announced even preliminary results showing that they’ve met that standard.

(Aside: “blinding” is really important in these trials. If subjects know they’re getting a placebo, they might be extra-careful to avoid exposure to the virus. This would artificially depress the number of cases in the placebo group. Conversely, if they know they’re getting the vaccine, they might be more reckless, increasing the exposures and cases in that group. In order for the results to be valid, we need all the subjects to behave the same.)

A faster option? There is a way to speed up this process: a “challenge” trial, where subjects are intentionally infected with the virus. The UK is preparing to start such a trial in January, first administering vaccines to healthy volunteers, and then exposing them to the SARS-CoV-2 virus about a month later. This is a far faster way to determine if a vaccine is working, but it creates serious ethical quandaries, because we don’t have a cure for the virus. If the world has an effective vaccine in January, I expect that the challenge trial will be cancelled. That wouldn’t be a bad outcome.

A new Russian Covid-19 vaccine looks promising, but did they fabricate some of their data?

Last week, a team of Russian scientists published the results of two phase 1/2 vaccine trials for a new Covid-19 vaccine developed in Russia. The study appeared in The Lancet, one of the world’s leading medical journals.

This vaccine has already received tremendous attention after Russian leader Vladimir Putin announced they would start administering it widely, before any phase 3 trials were under way. As I wrote last month, it’s not a good idea to skip these Phase 3 trials.

Nevertheless, the results from the early stage trials of both vaccines look quite good. Although the trials were small, with just 76 subjects, 100% of the subjects had a strong antibody response, and none of them had anything more than mild reactions to the vaccine. This suggests that both vaccines might be effective, although it’s too soon (after just 76 people) that it will be safe on a large scale.

There’s another problem, though.

Within 3 days of the paper’s publication, Enrico Bucci from Temple University described a series of apparent duplications in the figures presented in the Russian paper. He published his findings on his website as a “note of concern” that dozens of other scientists have signed.

I’ve read the paper and looked at all the figures, and it’s clear that something is wrong with the data.

Let’s look at one example to see what is going on. Here’s a small part of Figure 2A from the paper:

Each little column of dots shows a distinct group of 9 subjects, where the height of a dot indicates the level of antibodies found in that subjects. Notice that the 9 subjects in the red box (boxes added for emphasis) on the left have an identical pattern to those in the box on the right. These are completely independent subjects, and such a pattern is exceedingly unlikely.

It’s possible that this happened by chance, but then the problem is that this isn’t the only apparently duplication. Prof. Bucci identified at least 13 instances where sets of results are identical or near-identical between two different time points or two different sets of subjects. The other duplications look a lot like the one shown here.

The simplest explanation is that the data for some of the experiments were simply copied over from other experiments. As reported in The Moscow Times, the lead author of the study, Denis Lugonov, said there were no errors in the data. Because the authors of the Russian study didn’t provide their raw data, and The Lancet didn’t require it, other scientists can’t really check.

What are we to make of this? The details of the study are clearly explained, and the Russian vaccines use a design (an adenovirus modified to contain the SARS-CoV-2 spike protein) that is similar to other vaccines that so far seem safe and effective. Thus it’s quite possible that this vaccine will work–and it will be good for the world if it does. But the questionable data raise questions about whether the scientists behind this phase 1/2 trial have really done all of the experiments that they describe. The study concludes by noting that a phase 3 clinical trial with 40,000 participants is planned. Let’s hope that one yields positive–and genuine–results.

[Hat tip to Retraction Watch for drawing my attention to this study.]