by Michael D. Anestis, M.S.
Building off of yesterday's discussion on the importance of a systematic approach to diagnostic assessments, I would like to discuss a somewhat older article that I believe does a wonderful job of objectively demonstrating a similar point: the importance of utilizing empirically supported treatments for mental illness. In 2005, Kelly Cukrowicz, Ph.D. and colleagues published a study in Professional Psychology: Research and Practice detailing the impact of a switch to empirically supported treatments at an outpatient clinic, the Florida State University Psychology Clinic (listed on our EST clinics page as an excellent resource for mental health services).
Before explaining the structure and findings of this particular study, let me quickly review what is meant by the phrase "empirically supported treatments." ESTs are treatments that have data supporting their efficacy and/or effectiveness. In other words, rather than clinicians believing philosophically that a treatment works or citing a few examples that come to mind regarding the degree to which a treatment has utility in treating a particular set of symptoms, ESTs can actually demonstrate objectively that they produce results. Such data must be published in reputable peer-reviewed journals. There are several different levels of empirical support and treatments for which multiple independent research groups have replicated positive findings are at the highest level. For a more detailed description of which treatments meet the criteria for ESTs, we suggest Nathan and Gorman's A Guide to Treatments that Work
.
Although the degree to which individuals know about and clinicians practice ESTs has increased substantially in recent years, there remains a significant gap in knowledge and practice that has undermined a more universal transition to universal EST implementation. The knowledge gap is due, in large part, to a failure on the part of research oriented clinical psychologists to effectively market their products. It is no small wonder that, when nobody sells a product, nobody buys it. Compounding the lack of marketing for ESTs (at least psychosocial ESTs - effective pharmacological treatments are actually marketed quite well but are accompanied by equally prominent marketing for less effective pharmacological interventions), many individuals who espouse non-EST's market their beliefs and products quite prominently, thus perpetuating a misunderstanding of which treatments work and what help is available.
The practice gap is explained by a variety of issues. For some, the idea of switching to ESTs after being trained in a different approach is aversive. For others, there is a fundamental misunderstanding regarding treatment manuals that involves a false sense that there is no flexibility when a manual is used and the client is somehow dehumanized through this process. A complete list of the obstacles that impede the integration of ESTs into wide-scale every day practice is beyond the scope of this article, but suffice to say that these issues can often be resolved through a direct, honest, clear dialogue in which misinformation is readily corrected through education and conversation.
Back to the study at hand. The FSU Psychology Clinic implemented a shift in policy in 1998 that mandated the training in and utilization of ESTs. The authors thus decided that it would be useful to measure whether this shift actually had the desired effect. After all, if the driving force behind using ESTs is to ensure that clients receive the best possible treatment, we should be certain that ESTs actually accomplish that goal. From a sample of 2,250 patients between the years 1984 and 2001, 173 client files were randomly selected. 92 of these clients received treatment prior to the shift to ESTs and the other 81 received treatment after the shift.
The authors' primary hypothesis was that clients who received ESTs would exhibit significantly greater improvement as measured by the Clinical Global Impression (CGI) scale (Guy, 1976). Two to four therapists independently reviewed each client file and assessed the degree to which the client improved based on the CGI. Each reviewer based his or her assessment of improvement on progress notes, diagnostic and treatment summaries, and objective symptom measures such as the Beck Depression Inventory (Beck, Ward, Mendelson, Moch, & Erbaugh, 1961), the MMPI (Hathaway & McKinley, 1942), and the MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kraemmer, 1989). As hypothesized, individuals who received ESTs did, in fact, demonstrate greater improvement.
The results were more impressive than that one important finding, however. Not only was improvement greater in the EST group, but the effect size for this finding was extremely large (Cohen's d = .77). Whether or not a result is statistically significant depends, in part, on sample size. The larger the sample, the more likely a correlation will be significant - not because it is a stronger relationship but because the data is more likely to reflect a true relationship rather than a chance correlation (e.g., if you flip a coin four times and get tails three times, that's not significant, but if you flip a coin four hundred times and get tails three hundred times, you're more likely to wonder). Effect sizes, however, do not rely on sample sizes. Instead, they provide a measure of how strong a finding is relative to other findings. As such, a large effect size means that the difference between these two groups was substantial. Additionally, the average improvement for individuals in the EST condition was consistent with at least moderate symptom reduction whereas the average level of improvement in the non-EST condition was actually a slight worsening of symptoms. That's right, clients who received treatments not based upon data, on average, got worse.
To further test whether these findings actually reflected support for their hypothesis - that the use of ESTs leads to greater improvement - the authors were careful to examine alternative explanations. One possibility, they thought, might be that greater improvement was better explained by clients in ESTs attending more treatment sessions. This, however, was not the case. Clients in the EST condition attended significantly fewer sessions. In other words, not only did ESTs work better, they worked more quickly, thus reducing financial burdens and the length of time during which the client was suffering from acute symptoms.
A second possible alternative explanation the authors wanted to address was the potential that clients in the EST group improved more due to less severe initial symptomatology. Here again, the opposite was true. Prior to 1998, the FSU Psychology Clinic did not accept clients presenting with severe mental illness (e.g., suicidal patients, severely personality disordered patients). This policy changed at the same time ESTs were implemented and, as such, clients in the EST condition were the only members of the sample who possibly could present with a greater severity of mental illness.
A third possible alternative explanation for the greater improvement in the EST group was that they had a greater range of variability in severity, thus allowing for greater improvement simply due to a less restricted range. The authors thus performed a follow-up analysis in which only clients with, at most, moderate CGI severity at intake were included in either group. When they re-ran the analyses only using moderately ill clients, the results remained unchanged, with the EST group exhibiting significantly greater improvement through treatment.
So what does this study tell us? Contrary to the "Dodo Bird Hypothesis," all treatments are not the same. Treatments based upon empirical data produce better outcomes than do other treatments. Although ESTs have not been developed for all DSM-IV-TR diagnoses, some diagnoses have multiple ESTs available (e.g., cognitive behavioral therapy, interpersonal psychotherapy, and antidepressant medication for depression). Given the differential utility of different treatments - a point firmly supported by the findings in the Cukrowicz et al. (2005) study - it appears pivotal that clinicians utilize such treatments and that the field of psychology adequately market the availability of such treatments so as to bridge the gaps in knowledge and practice.
Importantly, I should note that both Joye and I worked for two years as therapists at the FSU Psychology Clinic. We did not work there during the time period from which client files were chosen or while this study was being written or published. We do not benefit financially or in any other fashion from publicity focused on the FSU Psychology Clinic. We are merely proud alumni of this wonderful source of mental health treatment.
Thus far on PBB, we have published a variety of articles on specific ESTs for particular diagnoses. Simply click on the empirically supported treatments link listed under "categories" on the right hand side of the screen to access all of these articles. Additionally, if you would like to learn more about the use of empirically supported treatments, we recommend the following products, all of which are available through our online store (manuals and description of these specific treatments are also available through our online store):
- A Guide to Treatments that Work
- Science and Pseudoscience in Clinical Psychology
- The Great Ideas of Clinical Science: 17 Principles That Every Mental Health Professional Should Understand
- Navigating the Mindfield: A Guide to Separating Science from Pseudoscience in Mental Health
- What's Wrong with the Rorschach? Science Confronts the Controversial Inkblot Test
- Seeing Both Sides: Classic Controversies in Abnormal Psychology
Mike Anestis is a doctoral candidate in the clinical psychology department at Florida State University.
Subscribe in a reader
