ClinicalTrials.gov, a great resource for patients, is being abused to market bad medicine

This wasn't supposed to happen.

Since 1997, the National Institutes of Health has maintained a database of clinical trials, ClinicalTrials.gov, that lists trials under way in the U.S. and throughout the world. It's an invaluable resource, providing a single source for patients trying to find where to get the newest experimental treatments, and for doctors and scientists looking to enroll patients in their trials.

In recent years, companies offering questionable stem cell therapies got the bright idea that they could describe their treatments as clinical trials, register them on ClinicalTrials.gov, and thereby get some free advertising. Most stem cell treatments are not FDA-approved, and many have little or no data supporting their effectiveness, but clinics can still register their "trials" on the NIH site, making it appear that they are supported and endorsed by the government. A new study by Leigh Turner, published this week in Regenerative Medicine, reveals the growing extent of this problem.

What's especially worrisome is that some stem-cell treatment clinics charge patients very high fees to participate in their "trials." Some patients (perhaps most) don't know that legitimate clinical trials virtually never charge fees.

Consider this example, reported last July by Emily Bazar at Kaiser Health News: California resident Linda Smith has knee osteoarthritis, and was looking for treatments that could restore her knees to health without surgery. She found a stem cell trial at ClinicalTrials.gov that was run by StemGenex, a clinic in La Jolla. The clinic promised it could inject stem cells into her knees to replace lost cartilage. When she inquired about signing onto the trial, Smith was shocked to learn that StemGenex wanted a $14,000 fee for her to participate.

StemGenex claimed that “The actual treatment is not part of the study protocol”: you pay for the treatment, they explained, and the study is merely a followup to see how you did afterwards.

What nonsense. I checked the StemGenex site today (a year after the quotes above), and they proudly boast that they are registered on ClinicalTrials.gov, and that

"Stem cell therapy for Osteoarthritis is being studied for efficacy in improving the complications in patients through the use of their own stem cells."

Hmm. This sure sounds like the study is about the treatment. StemGenex's site strongly suggests that their therapy works wonders:

"The goal of each stem cell treatment is to inject the stem cells into the joint to create cartilage (chondryte cells)."

Sounds good, right? If only it were true.

I've got bad knees myself, so I've been following the research on stem cell treatments for cartilage replacement for years. I would love to be able to get a simple injection that could repair my damaged cartilage. Alas, though, no one has yet developed an effective stem cell treatment for bad knees, although it is plausible, and legitimate trials are under way right now (here's one).

Unfortunately, the lack of evidence hasn't stopped clinics from offering stem cell injections right now, accompanied by all sorts of promises that are not backed by science. It's not just knee injections, either: this past March, Sharon Begley at STAT reported on three women who were blinded by stem cell therapy injected into their eyes.

Most patients think, mistakenly, that if a clinic offers stem cell therapy, it must have been approved by the FDA. That's not true–clinics offering these therapies don't have FDA approval, and they argue that they don't need it (which might be correct, but that's a topic for another day).

Patients also assume that trials listed on ClincialTrials.gov must have been approved by some government agency, but that's not true either. The site is a clearinghouse that uses the honor system, nothing more, to ensure that trials listed there are legitimate. If you read their Disclaimer (but who does?), you find that studies listed on the site are not necessarily funded by NIH or approved by the FDA.

Turner's study found 7 trials that openly state they charge patients to participate. At least they're honest about it. Turner found many more (including several run by StemGenex) that appear to charge patients despite not explaining their policy on ClinicalTrials.gov. For example, a stem cell trial by Cell Surgical Network plans to enroll 3000 patients and will charge each of them $6000 or more to participate. As Turner writes

"Cell Surgical Network uses its registered ClinicalTrials.gov study as a powerful marketing device. Press releases and the websites of the clinics that are part of this network emphasize that the study is registered on ClinicalTrials.gov."

NIH needs to start policing this site before the situation gets worse. Coincidentally, I know just where they can find the resources to do it. NIH just announced that it's about to start regulating all sorts of basic science studies as clinical trials, a move that will cause a "massive amount of dysfunction and paperwork," according to one MIT scientist. Rather than over-regulating basic science, NIH should devote those same resources to cleaning up and then continuously monitoring the ClinicalTrials.gov database.

What these stem cell clinics are doing is not a clinical trial, and advertising their services through ClinicalTrials.gov is reprehensible. For now, if a doctor or clinic tries to charge you to participate in a clinical trial, your best course may be to find another trial–and another doctor.

CRISPR gene editing controversy - does it cause unexpected mutations?

Just over a month ago, a short paper appeared in Nature Methods saying that the gene editing technique known as CRISPR-Cas9 has a big problem: it creates unexpected mutations all over the genome. This was startling news for a technique that has been hailed worldwide as a dramatic breakthrough, not only because it is the easiest gene-editing method yet invented, but also because it is (supposedly) very precise.

This new paper, by Kellie Schaefer and colleagues, found hundreds of mutations (in experimental mice) that weren't supposed to be there. The results contradicted earlier studies that showed CRISPR caused very few of these "off-target" mutations. One of the authors, Stephen Tsang, commented that

"We feel it's critical that the scientific community consider the potential hazards of all off-target mutations caused by CRISPR."

Not surprising, the resulting news headlines were gloomy. The stock in three companies trying to commercialize gene editing–Editas Medicine, Intellia Therapeutics, and CRISPR Therapeutics–all fell sharply.  (Interestingly, the stocks started falling on May 24, and bottomed out on May 31. The paper appeared online on May 30.) Scientists involved with these companies quickly responded, arguing that the study was flawed, but of course those scientists have a lot of money at stake.

Who was right? Well, a new paper by Caleb Lareau and colleagues, just released in the bioRxiv preprint repository, re-examines the same data and concludes that CRISPR is just fine. I've read both papers so you don't have to. Here's what seems to be going on.

The study by Schaefer et al. used CRISPR-Cas9 to create mutations in two mice (called F03 and F05), and then sequenced their genomes. They also sequenced the genome of a third mouse, called FVB. All three mice were supposed to be genetically identical.

Then they compared all three genomes to a "reference" mouse to find mutations. (Aside: this is something my own lab does all the time, so I know the techniques well.) They found over 1,500 mutations in each mouse (which wasn't surprising, because the reference mouse differs from their 3 lab animals), but they found hundreds more mutations in the two CRISPR-edited mice. That was the main surprise from Schaefer's paper, and it's the basis for their claim that CRISPR causes numerous off-target mutations.

I had a big problem with this claim even before reading Lareau's paper. Just TWO mice? That's a ridiculously tiny sample. But I digress.

Lareau et al. pointed out, correctly, that Schaefer's conclusion depends on the mice being genetically identical. But what if the two CRISPR mice (F03 and F05) were closer to each other than to the third mouse, FVB? (It's analogous to comparing two siblings with a first cousin, although these mice are much more inbred than any humans.) In that case, the result falls apart.

Fortunately, Schaefer et al. made all their data available (props to them for doing that), so Lareau could answer this question quite precisely.

It turns out that F03 and F05 are much closer to each other than either one is to FVB.  Lareau discovered that the two CRISPR mice share thousands of mutations that FVB doesn't have.

What does this mean? Lareau and colleagues conclude that the "unexpected" mutations in the CRISPR-edited mice were already there before the experiment began, and were not caused by gene editing. As they put it,

"the CRISPR-treated embryos most likely already harbored these private SNPs and indels prior to nuclease treatment whereas the control mouse did not."

In other words, it seems highly unlikely that CRISPR gene editing caused hundreds of unexpected mutations in these mice.

Even though CRISPR is being over-hyped right now, it is nonetheless genuinely exciting technology. Nature Methods was probably too eager to publish a controversial result, an all-too-common problem with big-name journals, and they seem to have done a poor job managing peer review. (Aside: I'd love to see what the reviewers said. Did they miss the obvious problems, or did the journal editors ignore the reviewers? I doubt we'll ever know.)

A final note: this kerfuffle illustrates the tremendous value of rapid publication through pre-print archives. Lareau et al.'s paper appeared a few days ago (July 5) on bioRxiv, along with all the data they used to support their arguments. We'll probably see a journal version too, but that will take months. Getting this paper out faster was a win for science.

(Postscript: two of the authors on the bioRxiv paper have financial interests in CRISPR technology companies, which they disclosed in the paper. I have no financial interests in any of these companies.)