by Michael D. Anestis, M.S.
On Monday, we wrote an article discussing a study published in JAMA and written by Jay Fournier and colleagues (2010) that called into question the degree to which antidepressants are effective for individuals with mild to severe depression (click here to read the article). The Fournier et al (2010) article received a lot of attention nationwide and, in fact, was the subject of several pieces in the New York Times over a series of days. Last night, the Times published an article that, in my opinion (and that of others who contacted me after reading the piece) severely distorts the methodology and results of the study (click here to read the Times piece). We do not ordinarily respond to every news piece with which we disagree, but I think this is a useful teaching point and a good topic for readers to discuss, as it impacts such a massive number of people.
The Times piece, written by Richard Friedman of Weill Cornell Medical College, made several claims about the Fournier et al (2010) study that I believe are highly questionable at best and I would like to address them here and see what you think.
1. Friedman stated "it contradicted literally hundreds of well-designed trials, not to mention considerable clinical experience, showing antidepressants to be effective for a wide array of depressed patients."
- There are good and bad elements to this statement. On the one hand, Friedman is right to caution us against drawing too many conclusions from a single study. An evidence-base is built up over time and our confidence in the results of any study must be tempered until they can be independently replicated in different samples numerous times. Fortunately, Fournier and colleagues (2010) emphasized this point themselves in the actual JAMA article. At the same time, this comment is a bit misleading in that it overlooks the fact that the vast majority of clinical trials for antidepressants do not, in fact, include a large number of depressed patients of varying severity. Additionally, "clinical experience," which Friedman cites as contradictory evidence, is not a good contrast for empirical data. As the work of Paul Meehl has so elegantly explained, we as humans are highly biased and error prone with respect to our ability to recall, organize, and analyze large amounts of retrospective information. Our experience as clinicians is thus not only generally limited to a small sample (remember, to run an effectively powered experiment, there must be a large number of participants), but is also so full of subjectivity and error as to render it useless as a measure of clinical outcomes over time. It's not that experience does not matter or tell us anything - it's simply that our own memories rarely if ever outperform actual empirical data.
2. Friedman cautioned against the use of a meta-analysis to decide this question, primarily because of the studies chosen for inclusion and because patient level data were obtained from only 6 studies (718 patients).
- As I have made clear here on a number of occasions recently, I also have a lot of reservations about meta-analyses, with some of my concerns paralleling those stated by Friedman. That being said, a closer look at his statement reveals some logical holes. First of all, the fact that only 23 studies fit the inclusion criteria - which are explained and justified in detail in the actual JAMA study - represents a flaw in antidepressant trials, not in the Fournier et al (2010) study. It is not their fault that most of the trials are not rigorous to fit within the confines of their study. Secondly, while it sounds as though only obtaining patient level data for 6 studies is a weakness, keep this point in mind: most meta-analyses obtain ZERO patient level data from the studies they analyze. Instead, meta-analyses typically rely on aggregate data (e.g., means, standard deviations) from the total sample of a published study rather than getting individual data from each participant and combining that with the data for participants from other studies, thereby allowing for a richer analysis. Basically, taking this approach, Fournier and his colleagues (2010) were able to expand the sample size from 6 (studies) to 718 (individual patients), which is large enough to reliably detect even very small effects. Still, I do agree with the general sentiment of using caution in responding to these findings on the basis of one study.
3. Friedman then discussed the decision on the part of Fournier et al (2010) to exclude studies that utilized a placebo washout period, stating that "many patients with depression - as many as 50 percent, in some studies - get better with no drug at all, just a placebo pill and attentive treatment by a therapist. For that reason, researchers often design their studies to exclude such people, to determine whether the drugs are working independent of any placebo response."
- This point is simply illogical. In a clinical trial for an antidepressant medication, the goal is to determine the extent to which the drug outperforms a number of other approaches, including a wait-list receiving no treatment, and a placebo group. Friedman's argument is that the best way to do this is to not include people who respond to placebos. This makes no sense. The best way to see if antidepressants outperform placebos is to compare people who received antidepressants to people who received placebos and to empirically measure whether the antidepressant group demonstrated significantly greater levels of improvements. If placebo accounts for the progress in the antidepressant group, there will be no difference in outcome between the two groups. By removing people from the study who respond to placebos, researchers create an artificially large gap between the responses of people who receive antidepressants and people who receive placebos. Their research question shifts from "are antidepressants more effective than placebos" to "not counting the people who respond to placebos, do people respond better to antidepressants than they do to placebos?" Essentially, the research team is removing the antidepressant's greatest opponent (the actual rate of response to placebos) and then testing to see whether it won the competition.
4. Friedman then critiqued the inclusion of analyses on only two antidepressants. He mentioned that antidepressants are not interchangeable and that, when the FDA conducted an examination of their efficacy, they analyzed over 300 trials with over 80,000 patients and approximately one dozen antidepressants.
- First off, the whole point of the Fournier et al (2010) study was to mention that the evidence-base for antidepressants as a treatment for mild to severe depression is, in fact lacking. The FDA did not analyze 300 trials that included a large number of individuals with mild to severe depression, they analyzed 300 trials that focused almost exclusively on very severely depressed individuals and excluded people who respond well to placebos. Such a study is entirely incapable of answering the question answered by Fournier and colleagues (2010). Secondly, although I agree that antidepressants are not all interchangeable with one another, Friedman seems inconsistent with this point. On the one hand, he criticizes Fournier and colleagues (2010) for making generalizations about the entire class, but on the other hand, he cited relapse rates and the evidence-base for antidepressants as a whole himself several times throughout the article. If we can not assume they are the same and need to analyze them all individually, then we should not speak about them as a single class when doing so makes them look good either. Certainly an analysis of each individual antidepressant would be more valuable than an analysis on only two. Hopefully these results will inspire researchers to conduct the types of trials necessary in order to make such analyses possible. Given the lack of financial incentive for doing so, I am doubtful this will occur.
5. Friedman mentioned that the real test of whether or not a treatment for depression is effective is relapse rates.
- I agree with this; however, Friedman did not mention that the relapse rates for psychosocial interventions (e.g., cognitive behavioral therapy) are significantly lower than those for antidepressants. Additionally, he did not mention that relapse rates for individuals who eventually discontinue their antidepressants (rather than remaining on them indefinitely) are significantly higher than those who receive CBT (and finish therapy after an average of 12-20 sessions). Click here to read John Ludgate, Ph.D.'s guest article on relapse in CBT.
Conclusions
Overall, I think that Richard Friedman did us a good service in reminding everyone that a single study is not a justification for a paradigm shift. At the same time, he did us a disservice by dismissing the results for reasons that make little to no sense rather than acknowledging their importance and putting forward the idea that they should serve as a wake-up call and a motivation to perform clinical trials better able to answer the question of whether or not antidepressant medications are effective for mild to severe depression. Every once in a while, a study comes along that surprises us and contradicts our strongly held beliefs. As scientists, we have a number of options in that situation. We can examine the evidence and put forth a logical rebuttal, we can incorporate the findings into our knowledge base and revise our own hypotheses, or we can simply deny the results are valid on the basis of false claims, emotional arguments, and human bias. In this case, I think Friedman chose an option most consistent with that third choice and, unfortunately, his action was granted a wide stage with a large audience.
If you would like to learn more about the topics discussed on Psychotherapy Brown Bag, we hope you will visit our online store for scientifically-based psychological resources.
Mike Anestis is a doctoral candidate in the clinical psychology department at Florida State University
Subscribe in a reader
