by Michael D. Anestis, M.S.
A little over a year ago, an article was published in the American Psychologist examining the efficacy of psychodynamic psychotherapy (Shedler, 2010). We reviewed that article here on PBB and then wrote another piece about some of the media attention that it was receiving. These articles prompted a fairly intense response from readership, which I think is great, as it exposed readers to a multitude of perspectives.
Ultimately, we decided that our efforts on PBB were not enough, so, along with PBB guest author Scott Lilienfeld, world reknown for his efforts to promote good science, we submitted a comment on the article to the American Psychologist and it was subsequently accepted and, just a few days ago, published (Anestis, Anestis, & Lilienfeld, 2011). We were not alone in our efforts, as three other comments were also accepted for publication, each of which took issue with specific components of the Shedler article. As is the case in any situation like this, the author of the original study was given the opportunity to reply to the comments and all of these materials were published together in the most recent issue of the journal.
That brings us to today's post. You might remember an earlier PBB post in which Brandon Gaudiano posted a follow-up to his comment/reply exchange with Lars-Goran Ost regarding the empirical status of Acceptance and Commitment Therapy (ACT). Dr.Ost never replied on our site, but we saw this as an opportunity to extend a published conversation further, bringing it into a more public forum in which a broader population would have access to it. Today, we will be trying this again by extending the comment/reply exchange we had with Dr.Shedler back onto PBB.
I wish there was a way I could give readers access to the actual journal articles themselves and our friend Daniel Krieglstein is working on such an effort through his journalwiki project; however, due to copyright laws, I can not provide you with a link to the articles themselves. What I will do instead is quote from the Shedler article, describe it as well as I can, and encourage you to find access to the articles themselves to ensure that you have the resources to develop informed conclusions. I will assert some opinions here, so the best way to develop your own thoughts on this piece will be to read the articles yourself before encountering my commentary.
Okay...on to the article. In all honesty, I was fairly taken aback by Dr.Shedler's response in a number of ways. I'm only going to cover his responses to our comment rather than all of the other authors, although I invite readers to expand the conversation to include those other components.
(1)
Let's start with this quote (p.152-153):
"Three of the four comments on my article appear to have the intent of reasserting the master narrative by creating a smokescreen of doubt and confusion (Anestis, Anestis, & Lilienfeld, 2011; McKay, 2011; Thombs, Jewett, & Bassel, 2011). Intentionally or not, they offer misinformation, omit crucial information, cherry-pick findings to create a misleading impression of the literature, or impose a disingenuous double standard with respect to what constitutes evidence."
First off, if you read our comment on his article, you'll note that we use a respectful and professional tone throughout. To each their own, I suppose.
Secondly, did we "have the intent" or was it "intentionally or not?" If you're going to make an accusation regarding an individual's intent, do it or don't. In this case, the latter would have been the better option due to the gross inaccuracy of the assertion, but I'll explain why more as we go along here.
My main issue with this sentence, as has been the case in numerous mainstream media pieces by this same author, is that it is not only inaccurate, but in fact a mirror opposite of reality. The Shedler (2010) piece was littered with misinformation and we used the entirety of our comment to discuss specific points of error. The American Psychologist limits authors of comments to eight total citations (we were lucky enough to be granted one extra one). Given that Shedler primarily discussed meta-analyses, which are comprised of a vast number of studies, there was no way for us to cover all of the errors in his article within our piece. We noted this in our comment and have covered many other errors in our PBB article. That being said, being accused of cherry-picking when you had to negotiate and luck out to receive room for even one extra citation is more than a bit disingenuous, particularly given that Shedler used the first page of his reply to write a philosophical piece about a "master narrative" skewing the views and behaviors of researchers and, in doing so, neglected to respond to 95% of our points about the evidence. Perhaps he was also at his cap on citations, in which case he was equally hamstrung. Either way, the accusation towards us is simply untenable.
(2)
The next point of contention in Dr.Shedler's reply had to do with our discussion of a meta-analysis by Leichsenring and Rabung (2008) in which those authors concluded that long term psychotherapy was superior to short term psychotherapy. In that meta-analysis, the authors made a computational error that resulted in an impossible effect size. Bhar and colleagues (2010) published a discussion of this point that also covered a number of other errors in that meta-analysis. Shedler mentioned in his reply that it was irrational for us to comment on that meta-analysis because the authors had published a correction a year and a half ago (Leichsenring & Rabung, 2009). He even used italics on the phrase "a year and a half ago," apparently to express his extreme chagrin with our decision. Here's the thing...we didn't choose to reference the 2008 study. You know who did? Wait for it.................Shedler himself! That's right, in his original article (Shedler, 2010), he devoted substantial text to detailing that precise reference. We simply replied to his text and examined the very article that he himself discussed. I seem to remember the phrase "double standard" being thrown around in the quote I listed above. Seems potentially relevant here, I think.
Another important point was raised here though. Shedler rightly pointed out that, when the authors recalculated the effect size, it was still large. This would be a more crucial point if that were, in fact, the only issue with that meta-analysis. On page 152, Shedler wrote:
"Also overlooked was the fact that CBT and other 'empirically supported therapies' were almost never tested against actual psychodynamic therapy - most of the research simply compared these treatments with no treatment, or with minimal, essentially sham treatments that were not designed to be seriously competitive approaches (what my colleague Drew Westen has called 'intent-to-fail' conditions, which are often misleadingly labeled in the research literature as 'treatment as usual.')."
Here's an interesting point to consider in light of this quotation. In the meta-analysis that Shedler was defending, the following treatments were considered "shorter term methods of psychotherapy" and thus used as comparisons to long term psychotherapy in the analyses that produced the effect size he touted:
- Nutritional counseling for anorexia nervosa or bulimia nervosa
- Treatment as usual - psychiatric care for borderline personality disorder (BPD)
- Treatment as usual - routine treatment for anorexia nervosa
- Treatment as usual - waitlist for BPD
- Treatment as usual - community for BPD with comorbid alcohol dependence
I wonder if Dr.Shedler's colleague Drew Westen still considers these "intent-to-fail" conditions when they are used in comparison to treatments favored by Dr.Shedler.
Interestingly enough, when long-term psychotherapy was compared to actual evidence-based treatments (e.g., dialectical behavior therapy for BPD), it failed to demonstrate superiority. Not surprisingly, this was not mentioned by Dr.Shedler in his reply to us...the same reply in which we were accused of cherry-picking results and presenting a misleading picture of the scientific literature.
(3)
Shedler next confronted us by expressing disagreement with our assertion that he should have included a study by Svartberg and Stiles (1991). His first point of contention was that the study was out of date. This would be a more powerful point if not for the fact that two of the studies included in meta-analyses that Shedler's conclusions were based upon in his original paper and which we discussed in our reply were from the early to mid-1980s. Others that we did not critique were even older.
His second point was that he did include this data, but through the lens of a re-analysis by Anderson and Lambert (1995). First of all, please note that this study came out four years after the one that was "out of date." Secondly, this was not a re-analysis of the Svartberg and Stiles (1991). They excluded five studies from that meta-analysis and included 11 additional ones. That being said, I can see how there is a degree of redundancy in citing both, even with the conflicting findings. All this being said, although the conclusions of Anderson and Lambert (1995) do not conform to those of Svartberg and Stiles (1995), they also don't conform to the conclusions of Shedler (2010). For instance, on page 509, Anderson and Lambert noted "we found no evidence that, in general, short-term dynamic therapy is better or worse than other forms of treatment." They did find that STDT outperformed CBT in the treatment of substance use disorders, however, they noted that this was based on one single study (Woody, 1983), which is not only nearly 30 years old....but is also a study that we specifically pointed out in our reply to Shedler and which was also included in other meta-analyses he cited in his paper (a blow to the rally cry that including Svartberg and Stiles [1991] would be redundant).
By the way, studies included in the Anderson and Lambert meta-analysis included the following publication years:
1989, 1984, 1983, 1990, 1981, 1990, 1983, 1980, 1986, 1987, 1991, 1979, 1989, 1983, 1983, 1975, 1985, 1989, 1988, 1991, 1979, 1989, 1993, 1982, 1987, 1984, 1988, 1979, 1983, 1975, 1987, 1983, 1987, 1990.
(4)
p.153 - "Unlike the other comments, the comment of Tryon and Tryon (2011) appears to be a sincere effort to engage with my arguments."
Again, professionalism and respectful language in scholarly journals works for some people and not for others. We each decide where we fit in that divide. If this were a blog post by Shedler, a different level of formality would make sense. Clearly I let my emotions and opinions fly at times on PBB. To me, I consider our effort to fill our comment with as many specific references to the data to be a sincere effort at engagement. I am less convinced that his response fits that description.
(5)
p.154 - "What disturbs me about the three other comments is not that the authors disagree with my conclusions but that they portray themselves as objective investigators who desire only to promote good science. I have little doubt that that is how they geniunely see themselves. People speaking from a dominant paradigm often assume that they are speaking obvious truths, while people in more marginalized groups tend to experience those in power as self-justifying, self-serving, and blind to important information that does not comport with their own worldview (much as academic psychologists experiences many of the medical psychoanalysits in the middle of the 20th century). From my perspective, the authors of these comments essentially offer tendentious arguments aimed at promoting an ideological agenda. I would go so far as to say that their writings betray a troubling disrespect for scientific evidence...from my angle of vision, they appear to value only evidence that supports an a priori agenda while ignoring, dismissing, or attacking evidence that does not. If so, this is not science, but ideology masquerading as science."
I've said it before and I'll say it again. It is difficult to be less anti-establishment than proponents of psychodynamic psychotherapy. This theoretical approach has had such a powerful grip on the field for so long that, even as its jargon has been removed from the DSM, it remains the go-to vision of psychotherapy for a large proportion of the population. A simple look at the data regarding forms of treatments utilized in "real world" practice would reveal that CBT is not the dominant force that Dr.Shedler's "master narrative" description makes it out to be (click here to read Dr.Lilienfeld's description of some such data).
Dr.Shedler's rant comes across like a campaign speech by the billionaire son of a career politician, proclaiming himself to be an outsider, a "real American," and an agent for change. The words sound compelling and, if true, would make for an impressive and virtuous cause. Sadly for the speaker, in this case, much like those of the hypothetical politician above, they are hollow. Ideology masquerading as science is, indeed, a horrific problem. Making absurd claims that Joye and I are engaging in such behavior and disrespecting scientific evidence is problematic enough, but to include Dr.Lilienfeld in a comment like that undercuts whatever credibility Dr.Shedler's comment could possibly have been intended to maintain. Dr.Shedler utilized nearly half of his article to talk about things other than evidence. Given an opportunity to discuss science, he instead chose to discuss us.
***
Before I sign off here, let me clarify a few points likely to come up in responses. First of all, I am not claiming that psychodynamic psychotherapy does not work. Although I have my concerns about it, all I am saying is that the conclusions of Shedler (2010) are not consistent with the evidence that he did and did not site on the matter. Second, I am not a "CBT guy." I like CBT in the sense that I support the use of all empirically supported treatments. That being said, I am fully aware that no treatment is perfect or works for everyone. Third, I have nothing against Dr.Shedler personally. I have never met him and, although I have been offended by his words and confrontational with his work, I fully believe that he is pursuing what he believes is a just cause. That being said, let's try to keep the comments focused on the data and the studies and not the authors themselves. Obviously I crossed that line here, but I did so in response to comments made about me, my wife, and my colleagues in a prestigious scholarly journal.
************
If you would like to learn more about the topics discussed on PBB, we recommend that you consult our online store for scientifically-based psychological resources.
Mike Anestis is a psychology resident at the University of Mississippi Medical Center and a doctoral candidate in the clinical psychology department at Florida State University
Subscribe in a reader
