by Michael D. Anestis, M.S.
On Wednesday, I covered a new article published in JAMA that called into question the utility of antidepressant medications for anything other than very severe depression. Today, I would like to switch gears to a distinct but related topic by covering two different meta-analyses that indicate that, while psychotherapy is certainly a strong treatment for adult depression, the degree to which this is the case might be overstated. Thank you to PBB guest author James Coyne for calling my attention to this research. My hope is that these articles will prompt some interesting and civil conversations, as the results are highly thought provoking.
Dr.Coyne's recommendation was to read and review an in press paper by Cuipers, van Straten, Bohlmeijer, Hollon, and Andersson looking at whether the impact of psychotherapy on adult depression has been overstated. In my search for this paper, however, I also came across another meta-analysis published by this group looking at the impact of psychotherapy on chronic major depression and dysthymia (Cuijpers et al., 2010). In both articles, Steve Hollon of Vanderbilt University, one of the pre-eminent cognitive behavior therapy proponents and researchers, is a co-author, so these results are not simply the work of researchers with an agenda aimed at discounting the value of psychotherapy. My plan is to quickly review each study and then to share some of my thoughts on what they mean.
Cuijpers et al. (2010)
In this first study, a meta-analysis published in Clinical Psychology Review, the authors wanted to compare the impact of psychotherapy, pharmacotherapy, and the combination of the two on chronic depression. Remember, depression by nature is episodic, meaning that it shows up, goes away, comes back again, and so on. A good treatment will break this cycle and prevent relapse. For some individuals, however, depression is a steady and prolonged battle. Some experience what is best thought of as dysthymia, a chronic sub-threshold depression that lasts for at least two years with no periods of two months or more during which the individual is without symptoms. For others, a full depressive episode is chronic or, more often, full depressive episodes are broken up by periods during which the individual is subthreshold, but far from symptom free. Regardless, an individual who fits into any of these categories faces a difficult path towards health and it is important to understand the degree to which various treatments can facilitate that process.
In order to investigate this issue, the authors searched for published studies that met a number of inclusion criteria. Specifically, in order to be included, each study had to be a randomized trial in which an active psychological treatment was compared to either another treatment, a control group, or a combined psychological and pharmacological treatment. Additionally, each study had to be conducted on adults with a chronic mood disorder as determined by a structured diagnostic interview. 16 studies met all of these criteria and were included in the analyses. The quality (e.g., how were participants randomized to groups, was a treatment manual used) of these studies varied widely.
Here's what they found:
Psychotherapy outperformed control groups, although the effect size was small (the actual effect varied depending upon which measure of depression was used). Pharmacotherapy outperformed psychotherapy, although here again the effect size was small. The combination of psychotherapy and pharmacotherapy outperformed either treatment on their own. The difference between combination treatment and psychotherapy was stronger than the difference between combination treatment and pharmacotherapy, although only four comparisons were available for the former, so that finding is fairly difficult to interpret. None of the treatment approaches resulted in higher drop-out rates than any of the others.
The authors also found that, in psychotherapy, the number of sessions was related to the strength of outcome, with a higher number of treatment sessions resulting in greater benefits (18 sessions was calculated to be optimal). You might remember from our prior article on sudden gains in treatment that other researchers have found that much of the improvement in psychotherapy (at least in cognitive behavioral therapy) occurs in early sessions. The authors of this paper concluded that, for individual with chronic depression, this may not be the case and that such individuals may actually improve more gradually throughout therapy.
The authors' overall conclusion was that psychotherapy, pharmacotherapy, and combination treatment are all useful approaches for chronic depression, but that the combination is likely the best option.
My thoughts:
I have covered my concerns regarding meta-analyses in prior articles (click here for an example), so there is no need to rehash those points now. That being said, these results were not particularly surprising. Chronic depression is notoriously more difficult to treat than are acute episodes and, as a result, therapies such as mindfulness-based cognitive therapy and CBASP have been developed specifically to treat these populations. In this meta-analysis, the authors did an admirable job of focusing on empirically supported treatments (although they did include supportive psychotherapy); however, I wish they had done a couple of things differently. First, as always, I would prefer that they compare specific forms of treatment to one another rather than simply examining "psychotherapy" in general as though all treatments are the same (we've covered the flaws of the dodo bird hypothesis elsewhere and I would recommend searching for that term on this site if you're interested in learning more about it). Secondly, I wish that a greater emphasis was placed on studying psychosocial treatments that are specifically aimed at treating chronic depression. At least one CBASP study was included in this meta-analysis, but there were no MBCT studies included.
A final problematic issue here is that the authors did not include any follow-up data. One of the reasons why psychosocial treatments are often considered to be a more cost-effective approach is that they result in substantially lower relapse rates. In other words, whereas both psychotherapy and pharmacotherapy can reduce acute symptoms, particular forms of psychotherapy (e.g., cognitive behavioral therapy, interpersonal psychotherapy) appear to be substantially better at keeping symptoms from returning in the future than antidepressant medication. Without follow-up data, this point is overlooked and the overall conclusions are thus substantially (and possibly spuriously) impacted.
Regardless, the overall conclusion that treatments in general are not as strong for chronic depression as they are for acute episodes is not altogether shocking and the fact that they are significantly better than control conditions is promising. More work needs to be done to improve the situation, but the situation is certainly not dire.
Cuijpers, van Straten, Bohlmeijer, Hollon, & Andersson (in press)
On to the study that Dr.Coyne recommended. Here, the same group of researchers conducted another meta-analysis, soon to be published in Psychological Medicine, in which they examined whether the quality of psychotherapy studies is related to how strong the results are from those studies. In other words, when studies are done properly to control for the types of things that can spuriously impact results, does psychotherapy still produce the same strong results?
In this analysis, the authors first examined studies of psychotherapy for adult depression to see whether or not they could be considered high quality. In order to be considered as such, the study had to involve the following:
- Participants met criteria for a depressive disorder as determined by a structured diagnostic interview
- A treatment manual was used
- The therapists were trained specifically in the treatment utilized in the study
- The degree to which the therapist administered the therapy according to protocol was checked and reported
- Intent-to-treat analyses were used, meaning that the authors did not only report on people who remained in treatment throughout the entire study
- At least 50 participants were included in the comparison between treatment and control
- Participants were randomized to groups by an independent third party
- The individuals who assessed outcomes (e.g., did the client improve?) were unaware of whether participants were in the treatment condition or the control condition
Anytime a study failed to mention that they met one of these criteria, it was assumed that they did not and that study was eliminated from the analyses. In total, 11 studies met all eight of these criteria (with 16 treatment comparisons). Of the 16 comparisons, 8 involved cognitive behavioral therapy and 13 used individual rather than group formats.
Here's what they found:
The overall effect size for studies including those that were not classified as high quality (d = .68) was high, meaning that, on the whole, psychotherapy was quite effective in the treatment of depression. The results were quite different, however, when they considered the quality of the studies. In the studies that were classified as high quality, the effect size was actually fairly small (d = .22 for high quality compared to d = .74 for other studies). In other words, psychotherapy in high quality studies results in less improvement than psychotherapy in lower quality studies. Importantly, this does not mean that scientific rigor renders psychotherapy less effective. Instead, it seems to indicate that when we ensure bias is removed, ratings of improvement are less substantial.
The authors were careful to check whether particular characteristics of the high quality studies (e.g., recruitment methods, target group, type of treatment) accounted for this difference, but their subgroup analyses did not support that notion. Similarly, the authors examined whether certain high quality studies produced stronger results than others based upon particular characteristics; however, none of those analyses were significant.
On the whole, the authors concluded that, while psychotherapy clearly produces improvement in adult depression, the strength of those improvements has been overestimated in past meta-analyses because studies of lower quality have been considered and these studies have artificially inflated results.
My thoughts:
Here again, I find myself a bit frustrated with the use of meta-analysis to make these types of determinations rather than looking at direct comparisons within single samples, but regardless of my reservations, meta-analyses remain popular and, as such, must be addressed. I do think that the authors raise some important notes of caution regarding the interpretation of clinical trial results. Specifically, they are spot on in telling us that we need to pay close attention to the methodological quality of a study before truly interpreting its results. Given that Steve Hollon signed off on these results, there is every reason to believe that they should be taken seriously, as he truly has nothing to gain by diminishing our view of psychosocial treatments given that he has done so much to champion to cause of empirically supported treatments.
Perhaps the biggest issue with this particular analysis is the small number of studies included (n = 11). This is a somewhat ironic issue given that individual studies with fewer than 50 participants were excluded from the meta-analysis on the basis that their low statistical power rendered them not "high quality."
A particular concern on a related front is the elimination of studies that did not specifically mention that they met all of the either inclusion criteria. While these studies may, in fact, have failed to meet the criteria, this is actually unknown, so the degree to which they actually represent lower quality studies is not clear.
As was mentioned with the first meta-analysis, a lack of follow-up data is an issue here too. At least as important as the question of how much better psychotherapy is than control conditions at reducing acute symptoms is the question of whether one is better than the other at long term improvement. Without even considering this question, it seems a bit hasty to discuss our view of the overall effect of "psychotherapy" (again, a general term referring to a heterogeneous group of treatments with differential abilities to impact particular diagnoses) on depression.
Overall, my thoughts on this meta-analyses can be summarized as follows: these results should highlight the importance of being cautious in interpreting treatment study outcomes and paying attention to methodological quality. That being said, these results should not take precedence over the results of comparative efficacy and effectiveness trials with follow-up data, which allow for the comparison of treatments and control conditions in a single sample rather than a zoomed-out approach in which two different studies are compared to one another.
One final point: the authors included a link to a website, www.evidencebasedpsychotherapies.org, at which you can access detailed information on this topic.
My hopes for this article
In a perfect world, readers will voice their thoughts in the comment section and an interesting conversation will ensue. In doing this, please try to remain respectful of the perspectives of others, no matter how vehemently you might disagree with what somebody says. The goal of PBB is to help as many people as possible make informed decisions on mental health issues. In order to do this, all sides of an argument should be given a fair shake and an emphasis should be placed on the evidence rather than our personal philosophies.
If you would like to learn more about the topics discussed on PBB, we hope you will consult our online store of scientifically-based psychological resources.
Mike Anestis is a doctoral candidate in the clinical psychology department at Florida State University
Subscribe in a reader
