Self-Efficacy Data in ECHO: Why & How We Use It
As we’ve written about on our blog before (here and here), using data to consistently measure and improve our programming is an essential part of the work we do here at ECHO-Chicago. One of the main ways we do this is through the pre- and post-series surveys that any past participants of our program know well. What we want to dive more deeply into today is more about the self-efficacy data we collect as part of those surveys: what it is, why it’s important to our program, and what the data we’ve collected over the years has shown about our program’s efficacy.
Self-efficacy questions document a provider’s confidence in their ability to practice and can span a range of things from confidence in prescribing antipsychotics to the ability to evaluate asthma severity to how well a provider feels like they can educate other providers about blood pressure management. In short, as our Research Manager Sandra Tilmon, MPH says, self-efficacy is a provider’s “belief that they can do a thing,” whatever that “thing” may be! In our program, self-efficacy questions are rated on 7-point scale, where providers self-evaluate their confidence from 0 (“no skills at all”) to 7 (“expert; able to teach others”). Tilmon says, “We aim to “bump” providers up from one level to the next, and for a 10% increase in SE (as a standard QI [quality improvement] goal), but regularly exceed that. For 844 paired surveys, we have a 1.13 point average increase – higher than our 10%/0.7 goal.” Though the average change varies for each series—as shown in the data below—we are proud that almost all of our series meet the 10% change threshold!
These tables show the self-efficacy change for our many series. This is updated frequently and can be found on our external dashboard.
While confidence in one’s ability is not the only thing that predicts behavior change, it is a crucial component of it. As our Founder and Director, Dr. Daniel Johnson, always says, the provider’s belief in their ability to act on the skills they’ve gained is just as important as the skills themselves. Research in other CME settings shows that self-efficacy and motivation to change are correlated with one another[1] and that it can often lead to behavior changes[2]. Thus, we can infer that by increasing self-efficacy we are also increasing motivation to change and behavior change in practice. Our surveys also measure intent to change clinical practice behaviors, as well as several other demographic and qualitative metrics.
There are a few other reasons that make clear why this type of data is so valuable to collect. First of all, self-efficacy data is used across the global ECHO movement and is a unifying metric for measuring growth and quality across hubs. This type of consistency across place and time helps ensure that as the movement grows, data is comparable. Second of all, self-efficacy—which is the third level of the AACME’s Seven-level Outcomes Model[3]— has a lower barrier to entry than, for example, patient outcome data drawn from an EHR. While the goal of our programming is still to improve patient health outcomes, diving into patient-level data can be time- and cost-prohibitive especially as an organization that works so closely with so many safety-net institutions. Doing complicated EHR data pulls would likely be an insurmountable barrier for many of our community-based partners around whom our program was built. Thus, it is far preferable to collect self-efficacy data, which is incredibly valuable on its own anyway, and maintain our program’s accessibility.
We are very proud of what our self-efficacy data shows and the ways it affirms the growth and motivation of our participants. As we see our program move day-in and day-out we were already confident in its impact, but to see it measured and tracked over time is especially helpful and meaningful!
[1] https://pubmed.ncbi.nlm.nih.gov/26115236/
[2] https://link.springer.com/article/10.1007/s40037-012-0012-5
[3] https://www.academycme.org/cappdfs/6.2%20The%20Seven-level%20Outcomes%20Model.pdf
Back to Updates