by Michael D. Anestis, Ph.D.
We at PBB are strong advocates for the concept of empirically-support treatments (EST's). As we have mentioned countless times before on the site, ESTs are treatments that have been shown through rigorous scientific research to produce the greatest results for the greatest number of individuals the highest percentage of the time for specific diagnoses. For example, exposure plus response prevention (EXRP) has demonstrated the greatest results in the treatment of obsessive compulsive disorder (OCD). Generally speaking, the strength of a treatment is demonstrated through effect sizes, which are statistical measures of the magnitude of an outcome and which allow for comparisons across studies. In other words, rather than simply saying whether or not a study found a statistically significant result, which can depend entirely upon how large their sample was and which can not truly be comared from one study to the next, authors who report effect sizes also give us an idea of how their results stack up to those of other works.
One issue with reporting only effect sizes, however, is that they only tell us the type of results produced on average by a particular treatment, with no clarification regarding the degree to which a particular client is likely to experience that average result. Many of the critics of ESTs argue that the treatments are not perfect and that some individuals truly benefit from alternative treatments. Proponents of ESTs do not argue against that point - no treatment is perfect for everyone, whether their ailment is psychological or physical - but stress the point that we have no systematic way of identifying who those folks are ahead of time and that, as such, we're forced to guess and the amazing work of the late, great Paul Meehl repeatedly and decisively demonstrated that when clinicians guess, we tend to perform worse than data....meaning that although intuition may lead us to correctly identify some patients that would benefit more from an alternative treatment, it will no doubt also lead us to incorrectly identify a number of people who do not fit into that group and, as such, our net result will be worse.
All of this brings me to an interesting study currently in press in Behavior Therapy and conducted by Oliver Lindhiem, David Kolko, and Yu Cheng of the University of Pittsburgh. In this article, the authors designed and tested what they refer to as the Probability of Treatment Benefit (PTB) chart, which calculates a simple to understand percentage that represents the likelihood that a particular individual will see a range of specific outcomes from a particular treatment. The study is preliminary and only looks at percentages based on one factor in a sample from one study testing one treatment - but it's a fascinating first step.
Essentially, what the authors did was look at a prior effectiveness trial for a modular, primarily CBT-based treatment for disruptive behavior disorders in children. With those data in hand, they looked at the likelihood of attaining particular outcomes given a client's pre-treatment levels of psychopathology. The authors chose to base their predictions off pre-treatment symptom severity due to past research indicating that it is highly related to outcome (e.g., Kazdin & Whitley, 2006). Importantly, the authors specified a number of outcomes they wanted to consider:
- Treatment response - To what degree do clients see significant changes in their level symptoms over the course of treatment? In other words, do they tend to exhibit substantially fewer or less severe symptoms than before treatment began?
- Treatment outcome - To what extent do clients meet particular criteria post-treatment? For example, do clients still meet diagnostic criteria for a particular diagnosis? Are they in the noraml range for symptom levels?
The authors noted that past research has shown that individuals with higher levels of pre-treatment symptoms are more likely to have stronger treatment response but poorer treatment outcomes. In other words, when you start with more symptoms, it makes sense that you would see greater change (e.g., regression to the mean, ceiling/floor effects) but it also makes sense that you would be less likely to end up with minimal symptoms (e.g., there is more room to change but more needs to change to reach a certain level; Reyno & McGrath, 2006).
I won't go into great detail regarding the methodology of this study as that extends beyond the scope of PBB and I'm a bit pressed for time, but please use the citations below if you would like to explore things to that level. Ultimately, the authors did in fact find that individuals with higher levels of pre-treatment psychopathology improved more during treatment but had worse outcomes. For example, individuals in the group with the highest initial severity levels had a 9% chance of being in the normal range post-treatment but a 64% chance of significant change in symptoms whereas individuals in the group with the lowest severity had a 68% chance of being in the normal range post-treatment but only a 21% chance of signficant change.
Ultimately, it is important to note that the authors only considered one variable (pre-treatment severity) as a predictor when many others contribute to outcomes (e.g., comorbid diagnoses), so future models need to take that into consideration. Additionally, the authors were only able to consider probability of particular outcomes for one treatment of one set of diagnoses in one sample, so the results need to be reproduced in separate samples before we can be confident in how the chart would work for this particular treatment and they need to be tested for other treatments before we can even begin to consider the results in those contexts. All of that being said, this marks an interesting first step. The end result could be an easy to understand complement to effect sizes and a way to help clients understand what the results of the scientific literature mean for them as individuals relative to the population as a whole.
************
If you are interested in learning more about this or other topics discussed on PBB, we recommend that you consult our online store for scientifically-based psychological resources.
Dr. Mike Anestis is a post-doctoral fellow with the Military Suicide Research Consortium.
Articles cited in this post:
Kazdin, A.E., & Whitley, M.K. (2006). Comorbidity, case complexity, and effects of evidence-based treatment for children referred for disruptive behavior. Journal of Consulting and Clinical Psychology, 74, 455-467.
Linhiem, O., Kolko, D.J., & Cheng, Y. (in press). Predicting psychotherapy benefit: A probabilistic and individualized approach. Behavior Therapy.
Reyno, S.M., & McGrath, P.J. (2006). Predictors of parent training efficacy for children externalizing behavior problems - a meta-analytic review. Journal of Child Psychology and Psychaitry, 47, 99-111.
Subscribe in a reader
