Field of Science

meditation, ADHD, and bad reporting

There's an excellent discussion over at Space City Skeptics that I wish I'd written myself. In it, Skepticpedi illustrates both the credulous reporting of the media on science, and a particularly poor study that was just reported. I say "reported" because it wasn't really published - the report was based on an article in an online education journal edited by graduate students.

What did the study (and the article) claim? Just this: that Transcendental Meditation (TM) was beneficial in the treatment of childhood attention-deficit hyperactivity disorder, also called ADHD.  The study did almost everything wrong - in fact, if you wanted to illustrate what *not* to do in a clinical study, this one would provide a wealth of examples:
  1. There were only 10 subjects, too few to make any statistically valid conclusions.
  2. There was no control group - all 10 subjects (children) were told to try TM techniques. By the way, TM basically consists of sitting and chanting a nonsense word to yourself, over and over.
  3. The results were based purely on self-reporting by the teachers.
  4. There was no "blinding" - all the teachers knew what the study was trying to show.
  5. The students were probably coached in what they were supposed to say.
  6. The headmaster of the school (all the students were at the same school) is a strong proponent of TM, and is on the board of the foundation that funded the study!
  7. The "scientist" (sorry, have to put that in quotes) who did the study is also on the board of the foundation that funded it. Yikes!
So basically, we have a small, terribly-designed, self-funded study by a proponent of TM who claims that she's produced evidence that TM helps treat children with ADHD. What's surprising here is that Reuters Health news service wrote a report on this (as did other news organizations, apparently). And by the way, the person who ran the study, Sarina Grosswald, has a strongly self-promoting website where she claims that she "recent directed a landmark research study" - this one! - and trumpeted the fact that it was widely reported in the media.  Calling your own study a "landmark", especially when it was self-labeled a "pilot" and published in an obscure journal, really takes some chutzpah.

So here's a big raspberry to Reuters for terrible medical news reporting. They can do better. And a big thumbs-up to Skepticpedi for calling them on it.

7 comments:

  1. Thanks for the shout out. This one hit a little closer to home than usual for me because I'm a pediatrician and I know how tough it can be for kiddos with ADHD. The last thing they need is some true believer selling them false hope, or taking credit for something their technique had nothing to do with. The kids in the study were on meds for pete's sake, and it wasn't controlled for at all.

    Skepticpedi

    ReplyDelete
  2. Meditation & ADHD has been studied more carefully.

    Zylowka, et al. (2008). Mindfulness meditation training in adults and adolescents with ADHD. Journal of Attention Disorders, 11, 737-746.

    reviewed by David Rabiner at Sharp Brains:

    http://www.sharpbrains.com/blog/2008/05/22/mindfulness-meditation-for-adults-teens-with-adhd/:

    - Summary and Implications -

    Results from this study indicate that mindfulness meditation training may be a beneficial complementary treatment approach for adolescents and adults with ADHD. Positive findings include: 1) the absence of any reported adverse events; 2) highly favorable ratings of the treatment by participants; 3) reductions in self-reported ADHD symptoms reported by over three quarters of participants, even though the majority were already being treated with medication; 4) significant improvement on several of the neuropsychological measures; and, 5) reductions in depressive and anxiety symptoms for the adults.

    The authors are appropriately cautious in discussing their findings and suggest that the study supports the "...feasibility and potential utility of mindfulness meditation in at least a subset of adults and adolescents with ADHD." They are careful to note, however, that this was a pilot study with a small sample, and that the reported pre-post changes in behavioral and neurocognitive measures should be "...considered exploratory given the absence of a control group and reliance on self-report measures of psychiatric symptoms."

    Also see the Mindful Awareness Research Center (MARC) at UCLA

    http://marc.ucla.edu/.

    ReplyDelete
  3. Liz: did you read the post by Skepticpedi, or my comments? Because your summary - and the article you linked to - provide another textbook case of a poorly done study:

    1. Very few patients - only 25
    2. A high drop-out rate (8 of the original 33 who signed up
    3. Results were self-reported! Which means no "blinding", allowing for all sorts of bias. In this case, subjects who believed that meditation worked are obviously likely to report some benefit.

    You write as if the self-reported "highly favorable" ratings of the study somehow are scientifically valid. They're not. Treating a patient well is likely to lead to favorable ratings - and of course it's nice to be nice to your patients - but it doesn't make for a valid study.

    I think the Zylowka et al. study just gives us another example of how to design a study that gets the results you want, rather than getting at the truth.

    ReplyDelete
  4. That's funny Steven, I didn't get that from her post at all. There were a lot of disclaimers in it. Perhaps you need to meditate a little before you post!

    - Jeff

    ReplyDelete
  5. Hi Steven,

    I am posting with regards to your comments on the mindfulness & ADHD study (Zylowka) ... Did you read the paper?

    I get the impression not. The authors main conclusion was a larger study is required, and warranted because of these findings.

    If you had read the paper, you would have also noticed that the assessment included the Attention Network Task (ANT). The ANT is a computer-based task (not self-report as you claim) - one that was developed by Posner.

    Finally, Mindfulness Meditation Training isn't a crackpot theory (as your dismissiveness of it suggests), it's heavily researched (including neuroimaging data) in relation to the positive effect it has on various psychopathologies.

    It's all very well having a blog discussing pseudoscience, but I think you should be more responsible in the future - you were overly critical of a technique that is already widely accepted and used in clinical practice.

    Maybe you should stick to discussing topics that fall in your area of expertise.

    Joe Butler

    ReplyDelete
  6. Anonymous: I read enough of the Zylowka study to stand by my comments. It's a poorly-controlled, small study, as I wrote originally. Like many people promoting their own work, the authors state that "a controlled clinical study is warranted." That's standard fare for pseudoscience - produce a vague result, claim it suggests your vague idea has merit, and ask for money to do more research. This study doesn't prove anything and it certainly doesn't convince me that they deserve more funding.

    ReplyDelete
  7. Hey Steven, I found this post looking for similar studies done with more rigor. Since you posted this have any crossed your path which might stand up to closer scrutiny?

    ReplyDelete

Markup Key:
- <b>bold</b> = bold
- <i>italic</i> = italic
- <a href="http://www.fieldofscience.com/">FoS</a> = FoS