by Joye C. Anestis
One of the primary arguments raised against the categorization of empirically-supported treatments (ESTs) is a concern that the studies used to determine empirical support do not generalize to real-world therapy situations. The primary studies required to classify a treatment as empirically-supported, known as efficacy studies, are randomized controlled trials (RCTs). RCTs randomly assign participants into treatment or a control group. The therapists in these trials are highly trained in the therapy and strictly adhere to treatment manuals. Individuals generally cannot participate if they have other diagnoses besides the primary one being treated. The purpose of these RCTs is to determine internal validity, to determine if the symptom reduction in the treatment group can be attributed to the treatment. It has been theorized that one of the reasons ESTs are underutilized in real-world clinical practice is that these RCT findings will not generalize to real-world situations (Westen, Novotny, & Thompson-Brenner, 2004). Clinical practice in the real world is often much messier than the one created in an RCT. Comorbidity of diagnoses (i.e., clients having more than one mental illness) is the rule rather than the exception (Clark, Watson, & Reynolds, 1995). Therapists are not always experts in specific ESTs, and pragmatic issues might hamper the faithful administration of an EST.
Luckily, another type of study aims to address this very issue. Known as effectiveness studies, these studies focus on the external validity of their findings, with the overall goal being to determine if results from RCTs represent results in the real world. Effectiveness studies are generally conducted at community mental health clinics, using the staff therapists administering ESTs as they normally do. Participants in these studies are community clients presenting at the clinic. Thus, effectiveness studies often involve routine service provided to consumers.
On PBB, we often talk about the results of RCTs in the treatment of various mental illnesses, something we will always continue to do. So far, we have written about interpersonal and social rhythms therapy for bipolar disorder, exposure for specific phobias, EX/RP for obsessive-compulsive disorder, CBT for generalized anxiety disorder, DBT for borderline personality disorder, CBT for PTSD, and cognitive-behavioral group therapy for social anxiety disorder. And we have many many many more to cover! However, it's helpful to stop every once in a while and determine if all of these results can be replicated in real world clinical situations. Unfortunately, for reasons I have never been able to understand, there are far fewer effectiveness studies conducted than efficacy studies. This is a real shame, as the effectiveness arm of the EST campaign is essential. However, in the past few years, the cumulative amount of effectiveness studies has grown enough to warrant a couple of studies looking at the aggregate effect size of various treatments.
One common way to compare outcomes from effectiveness studies to those from RCTs is to use a benchmarking strategy. Hunsley and Lee (2007) used a benchmarking strategy by identifying the best RCT (or a meta-analysis) for each mental illness. Think of it as comparing effectiveness studies to the "gold standard" results in more rigorous studies. Hunsley and Lee (2007) used the percentage of clients who completed treatment, percentage of client who were improved at treatment termination, and effect sizes as their standards for comparison. They then gathered effectiveness studies for adult and childhood/adolescent conditions. The overall results found improvements in effectiveness studies comparable to those from efficacy studies for both childhood/adolescent and adult disorders. Clients completed treatment at a rate higher than is typically found in psychotherapeutic outcome studies and (with few exceptions) at a comparable rate to the benchmark study. Almost all of the effectiveness studies had outcomes comparable to or greater than the benchmark study. It's important to note that these results come from a wide variety of treatment providers, such as inpatient units, specialty clinics, independent practices, outpatient clinics, and community clinics. Specific treatments tested:
- Cognitive therapy and CBT for depression
- CBT for panic disorder/agoraphobia
- CBT for generalized anxiety disorder
- CBT for obsessive-compulsive disorder
- CBT for social phobia
- CBT and IPT for childhood/adolescent depression
- CBT for childhood/adolescent anxiety disorders
- Parent training for disruptive behavior disorders in children
- Multisystemic therapy for adolescent conduct disorder
Obviously, there are a lot of mental illnesses missing from that list, but it's a start. Another interesting aggregate study of effectiveness literature is a very recent meta-analysis by van Ingen, Freiheit, and Vye (2009). They conducted a meta-analysis of effectiveness studies for CBT for anxiety disorders. van Ingen and colleagues found that cognitive-behavioral interventions for anxiety disorders (in this study, OCD, panic disorder with or without agoraphobia, social phobia, PTSD, and GAD) were associated with significant decreases in anxiety symptoms (mean variance-weighted effect size = 1.35). These gains were maintained at follow-up (mean variance-weighted effect size = 1.14). Cognitive-behavioral interventions for anxiety disorders also significant decreased concurrent depression symptoms at post-test (mean variance-weighted effect size = 0.96) as well as follow-up (mean variance-weighted effect size = 1.12). These results are comparable with meta-analytic studies on RCTs for cognitive-behavioral interventions for anxiety disorders.
Taken, together these two studies begin to verify the assumption long held by proponents of ESTs: that results from RCTs can generalize to treatment in the real world. This is an important message to send out into the psychotherapy community. ESTs, which time and again show documented and robust outcomes in clients, do work in regular clinical practice as part of routine clinical care.
If your are interested in learning more about empirically supported treatments, we recommend exploring our online store, which includes a variety of empirically supported treatment manuals aimed both for therapists and clients. Some of these books are self-help oriented, some explain the theories behind certain treatments, and others actually detail, step-by-step, how to administer particular treatments.
Joye Anestis is a doctoral candidate in clinical psychology at Florida State University.
Subscribe in a reader
