by Michael D. Anestis, M.S.
I try to keep a calm stance towards the topics we cover on PBB. Obviously, Joye and I have our own beliefs about things, but we try to keep our opinions out of our writing to the degree that such a move is possible and to base all of our comments on empirical evidence. In doing this, we try to keep emotion out of the picture and to therefore make it easier to facilitate civil conversations amongst readers. That being said, occasionally a story takes hold in the media that simply makes me incredibly angry. When the story is prompted by ignorance (e.g., a journalist with no training in data analysis writing on a topic with which he or she is unfamiliar), I find it relatively easy to turn that anger into frustration and understanding and to channel that into a calm discussion of the facts, thereby debunking myths and errors. When the story is prompted by a willful misrepresentation of the evidence and an open effort to distort reality through a highly restricted discussion of data based purely upon highly flawed studies, on the other hand, my response is a bit more harsh. All of this being said, today I would like to reflect on an article posted on a number of websites, including that of the American Psychological Association, an organization more than capable of looking at all of the data and accurately describing the facts (thanks to PBB guest author Dr. James Coyne for alerting me to this and making my topic choice for the day that much easier!).
This article is essentially an advertisement for the meta-analysis on psychodynamic therapy written by Jonathan Shedler and published in the American Psychologist that we discussed recently on PBB (click here for our coverage of the piece as well as free access to the journal article itself). Now, keep in mind as you read my article today that I have nothing against Dr.Shedler as a person or psychodynamic therapy as a general concept; however, I do have strong negative opinions about sloppy interpretations of data and misrepresentations of facts and the article covered today is weighed down heavily by both of those things.
Before reading my discussion of the article, you might find it useful to read it for yourself so that you can form your own impressions first. Click here to read it on the APA website and here to read the same text on Health Canal (you can leave comments on the article at Health Canal).
My approach today will be to quote the text of the article (in bold and italics) and then to reply below the quote.
***
"Psychodynamic psychotherapy is effective for a wide range of mental health symptoms, including depression, anxiety, panic and stress-related physical ailments, and the benefits of the therapy grow after treatment has ended, according to new research published by the American Psychological Association"In just about any news article, the first sentence (and the headline) will be an attention grabber and that's fine. The thing is, the claims put forth in this sentence are, in large part, based upon horrifically flawed data. Unfortunately, the vast majority of readers of that article are unaware of this fact and will never encounter those data. As such, their lasting impression of this issue is an unfounded claim. When discussing science, this is a dangerous approach. When the APA is publishing material on its site, such a non-scientific approach is highly unimpressive.
***
“The American public has been told that only newer, symptom-focused treatments like cognitive behavior therapy or medication have scientific support,” said study author Jonathan Shedler, PhD, of the University of Colorado Denver School of Medicine. “The actual scientific evidence shows that psychodynamic therapy is highly effective. The benefits are at least as large as those of other psychotherapies, and they last.”As with the sentence discussed above, I understand that including quotes by the authors of the article is a useful tool, but here again, a conclusion is stated and a passing reference is made to data...but the reader is left to believe that they are high quality data that produce irrefutable results. As we discussed in detail in our original coverage of the Shedler piece, this is so far from the truth that it would be difficult to overstate.
***
To reach these conclusions, Shedler reviewed eight meta-analyses comprising 160 studies of psychodynamic therapy, plus nine meta-analyses of other psychological treatments and antidepressant medications. Shedler focused on effect size, which measures the amount of change produced by each treatment. An effect size of 0.80 is considered a large effect in psychological and medical research. One major meta-analysis of psychodynamic therapy included 1,431 patients with a range of mental health problems and found an effect size of 0.97 for overall symptom improvement (the therapy was typically once per week and lasted less than a year). The effect size increased by 50 percent, to 1.51, when patients were re-evaluated nine or more months after therapy ended. The effect size for the most widely used antidepressant medications is a more modest 0.31. The findings are published in the February issue of American Psychologist, the flagship journal of the American Psychological Association.
The eight meta-analyses, representing the best available scientific evidence on psychodynamic therapy, all showed substantial treatment benefits, according to Shedler. Effect sizes were impressive even for personality disorders—deeply ingrained maladaptive traits that are notoriously difficult to treat, he said. “The consistent trend toward larger effect sizes at follow-up suggests that psychodynamic psychotherapy sets in motion psychological processes that lead to ongoing change, even after therapy has ended,” Shedler said. “In contrast, the benefits of other ‘empirically supported’ therapies tend to diminish over time for the most common conditions, like depression and generalized anxiety.”
First off, the writer of this article should be applauded for including actual data here and attempting to explain the meaning of effect size. Most mental health articles choose to skip such information, which is a disservice to the reader. That being said, here is where the use of meta-analysis caused some real problems. The numbers cited by the author sound highly compelling, but the results reflect poor data and low quality studies included in Shedler's meta-analysis. If anyone takes the time to look at the actual results of the studies included in Shedler's work, they'll see a completely different picture and, again, I encourage you to read our earlier article on this topic to see precisely what I mean here (I describe the actual studies themselves in detail). I realize that my opinion on meta-analysis is not universally accepted, but it would be difficult to argue against the point that many of the studies included in Shedler's study were of low quality and that the results directly comparing empirically supported treatments to psychodynamic therapy actually directly contradicted his conclusions.
Moving beyond this point, Shedler referred to these studies as the "best available scientific evidence on psychodynamic therapy." That's fine...except that this evidence is of poor quality. If this is the best that is available (an arguable assertion), than what is available is not good enough.
The claims of follow-up results are also not supported by the evidence. That being said, click here to read a guest article written by John Ludgate, Ph.D on relapse in cognitive behavioral therapy (CBT). That text will provide you with a more thorough understanding of what we know about this topic.
***
“Pharmaceutical companies and health insurance companies have a financial incentive to promote the view that mental suffering can be reduced to lists of symptoms, and that treatment means managing those symptoms and little else. For some specific psychiatric conditions, this makes sense,” he added. “But more often, emotional suffering is woven into the fabric of the person’s life and rooted in relationship patterns, inner contradictions and emotional blind spots. This is what psychodynamic therapy is designed to address.”This argument is one that consistently makes me irate. It does so for a number of reasons. First, it belittles the importance of symptoms. Here's the thing, symptoms like panic attacks, suicidal ideation, the inability to experience pleasure (anhedonia), hopelessness, non-suicidal self-injury, binge eating, substance withdrawal, and antisocial behavior are actually quite important. Helping a client to no longer experience those things is no small feat and nobody benefits when we act as though this is not the case. Second, he implies that empirically supported treatments such as cognitive behavioral therapy ignore everything except the symptoms listed in the DSM. There is no evidence that CBT does this or that psychodynamic therapy does it any less. It is simply a talking point, repeated by those who oppose the EST movement often enough that people have come to believe that it is true. Psychology is a science and, as such, our conclusions need to be founded upon evidence. When people simply make claims like this without any sort of support and we take them at their word for it, the entire foundation of the field collapses inward and those most in need of help - the millions of people suffering from mental illnesses - are harmed. As it turns out, EST researchers examine the impact of treatment on a vast array of outcomes unrelated to DSM symptoms, by the way, and a quick search through our articles on these treatments will provide you with numerous examples of this.
Additionally, the phrasing used by Shedler lumps proponents of ESTs in with the pharmaceutical and insurance industries, which are highly unpopular with most people. As such, scientists are suddenly the "bad guys," pushing an agenda upon the people whereas psychodynamic therapists are the anti-establishment offering freedom from oppressive interventions. The thing is, it's hard to be more closely associated with the establishment than psychodynamic therapy, which is such a popular conceptualization of mental illness and psychotherapy that it hard to find any media representation that takes any other approach. This group had so much control over this field for so long that the initial two versions of the DSM used their jargon and directly asserted that mental illnesses were best thought of in those terms. Empirically supported treatments are not motivated by profits (by the way, they take less time and cost less money than most psychodynamic approaches), nor are they associated with unpopular industries. They represent the belief system of scientists, who rigorously test their theories through systematic investigations.
The bottom line is, whether or not "more often, emotional suffering is woven into the fabric of the person's life and rooted in relationship patterns, inner contradictions, and emotional blind spots," the treatment that makes those symptoms disappear and thereby improves the individual's quality of life is the better choice. If a particular set of symptoms is treated and the client still has unresolved problems that he or she wants to address, practitioners who utilize ESTs are more than happy to address them. Nobody is kicked out of therapy or forced to ignore their own problems - instead, rather than the old school psychoanalytic preference of charging hundreds of dollars per session for multiple sessions per week over a period of several years, the newer, evidence-based approaches specifically target the issues that prompted the individual to come in for treatment and address them in an effective, time-limited manner. EST's simply prioritize things like reducing suicide risk as quickly as possible and I'm not certain I understand the counter argument to that priority.
***
Shedler also noted that existing research does not adequately capture the benefits that psychodynamic therapy aims to achieve. “It is easy to measure change in acute symptoms, harder to measure deeper personality changes. But it can be done.”This comment mystifies me and it's a great example of how our own biases can cause us to put forth arguments based upon contradicting points. Up until this point of the article, we have been led to believe that the evidence supports psychodynamic therapy. Still, there is a substantial research base left untouched (or misrepresented) in the Shedler analysis, so an answer is needed. What answer is offered? Evidence does not capture psychodynamic therapy. So....first they say look at all of this incredible supporting evidence and then they attempt to disarm their opponents by pointing out that evidence in general is not useful. In other words, let me shine a light on bad data that appears to support my case but simultaneously disavow data so that when people point out all of the evidence against my conclusions, I can say that it only contradicts me because it is not capable of seeing the truth.
This is the antithesis of science. You can not simply shine a light on the results that (appear to) support you and disregard the results that contradict your conclusions. Either science is good or it isn't (it is, by the way)...pick one.
***
The research also suggests that when other psychotherapies are effective, it may be because they include unacknowledged psychodynamic elements. “When you look past therapy ‘brand names’ and look at what the effective therapists are actually doing, it turns out they are doing what psychodynamic therapists have always done—facilitating self-exploration, examining emotional blind spots, understanding relationship patterns.” Four studies of therapy for depression used actual recordings of therapy sessions to study what therapists said and did that was effective or ineffective. The more the therapists acted like psychodynamic therapists, the better the outcome, Shedler said. “This was true regardless of the kind of therapy the therapists believed they were providing.”I discussed the flaws of this analysis in great depth in our original piece on this topic, so rather than rehash them here, I'll again encourage you to read the original article. The studies upon which Shedler's conclusion here is based are so flawed it is actually mind-boggling. He himself said in the original paper that "qualitative analyses of the verbatim sessions transcripts suggest that the poorer outcomes associated with cognitive interventions were due to implementation of the cognitive treatment model in dogmatic, rigidly insensitive ways by certain of the therapists."
***
Before concluding today, I want to provide you with another link, brought to my attention through an email sent though the listserv of the Society for a Science of Clinical Psychology. In this article, published in the LA Times (click here to read it), the results of the Baker et al (2009) report on the use of science in clinical psychology, are debated (click here for our coverage of the Baker et al report). This is a distinct but related issue and I wanted to call attention to three quotes from the LA Times article to drive home my point for today.
Quotes number 1 and 2 are from Drew Westen, a vocal critic of the EST movement. He referred to those who favor the use of ESTs as "largely people who not only don't practice themselves -- and therefore have no idea what would be relevant to practice -- but have a tremendous disdain for people who do practice." He also said that "[Cognitive-behavior therapy] is deliberately designed to ignore any relevant features of the personality of the individual."
Westen's first points, that EST supporters do not practice, that practicing is required in order to understand therapy, and that EST researchers have disdain for practitioners are, quite frankly, falsehoods. Many researchers are active clinicians. In fact, I would bet heavily that the proportion of researchers who practice is substantially higher than the proportion of clinicians who conduct and read research. Additionally, whereas all researchers are trained in therapy as part of graduate school and internship, not all clinicians are trained in research. If experience is required to understand one, than aren't non-scientific clinicians the only ones incapable of understanding the entire picture? Finally, upon what evidence does he base the claim that researchers have disdain for clinicians and, along those lines, what evidence does he have that clinicians have less disdain for researchers?
Westen's second point, that CBT was designed to ignore personality, is absurd. CBT is designed to address aspects of mental illnesses that have been shown to be common across individuals in order to maximize symptom relief; however, there is plenty of flexibility to work with the client as an individual. Regardless, given than CBT and variants of CBT (e.g., dialectical behavior therapy) have been shown to be effective in the treatment of personality disorders, it seems a bit off base to say that personality is not addressed in CBT.
Quote number 3 is from Michael Lambert, who said "I don't care what psychotherapy the person is getting. I care whether they're responding." He said this in an effort to point out that proponents of ESTs care more about providing a particular treatment than they do about clients responding to treatment. This, again, is absurd. The entire premise of the EST movement is that certain treatments have been shown, on average, to produce better results for particular diagnoses. These treatments are thus considered the best choice; however, nobody believes everyone will respond in the same manner to the same treatment and, as such, assessment is required in order to ensure that improvement is happening. Because EST proponents believe in this so wholeheartedly, an enormous research base has developed enabling us to better understand the impact of treatment. Opponents of EST's on the other hand, eschew assessments and, as such, have no idea the degree to which their treatment choice is effective. So, ironically, in his attempt to criticize the EST movement, Lambert actually explained one of the primary reasons why ESTs are so important. He also sheds a light on why the research base upon which Shedler built his case is so flawed.
***
So...what's my overall point today? Quite simply, the point is that when we hear somebody say something is a certain way, we need to always ask how they arrived at that conclusion and, when possible, we need to examine the evidence ourselves. In a meta-analysis like the one conducted by Shedler, it is easy for the authors to paint a picture that supports their point and to make very compelling statements that sound intuitively profound. If we do not look at the evidence underlying their conclusions, we are vulnerable to falling into traps. The claims made by people like Shedler, Westen, and Lambert are not a malicious attempt to mislead the populace, but they represent a sloppy, non-scientific approach to understanding mental health and psychotherapy. The media stories covering such claims result in a form of deception that makes me remarkably upset. If we simply listen to their words or the descriptions of journalists who write about them, we will not see things accurately. When organizations like APA post this type of thing on their website, they make the problem even worse.
Here on PBB, I likely say things readers disagree with on a daily basis. The thing is, I will always provide you with the citations upon which my points are based and will openly discuss the data with you. If you know of data I overlooked that calls my point into question, I encourage you to mention it in the comment section so that we can all have a civil discussion about these things. The next time I change my mind on an issue won't be the first, but this only happens when I am made aware of evidence that makes my previous position a worse reflection of reality than an alternative position. In the meantime, please, when you read about topics like this, make sure that what the article says is actually supported by valid evidence.
************
If you would like to learn more about the topics discussed on PBB, we hope you will consult our online store for scientifically-based psychological resources.
Mike Anestis is a doctoral candidate in the clinical psychology department at Florida State University
Subscribe in a reader
