J Community Psychol. 2021 Nov 19; doi: 10.1002/jcop.22749. Epub 2021 Nov 19.

Exploring perceptions of implementation practice capacity in community-based behavioral health organizations.

Journal of community psychology

Enya B Vroom, Oliver T Massey, Zahra Akbari, Skye C Bristol, Brandi Cook, Amy L Green, Bruce L Levin, Dinorah M Tyson, Micah E Johnson

PMID: 34797922 DOI: 10.1002/jcop.22749

Abstract

Community-based organizations (CBOs) must have the capacity to adopt, implement, and sustain evidence-based practices (EBPs). However, limited research exists examining CBOs' ability/capacity to implement EBPs. The purpose of this preliminary study was to investigate how staff of CBOs perceive implementation practice capacity, determine factors needed for adequate capacity for implementing EBPs, and examine which perspectives of capacity are shared across organizational levels. Ninety-seven administrators and practitioners of CBOs were surveyed using the Implementation Capacity Survey, which examines perceived importance, presence, and organizational capacity of the CBO in nine implementation practice areas (IPAs) (e.g., leadership). Results revealed participants rated IPAs on the importance scale higher than IPAs on the present scale. Presence and organizational capacity scales were strongly correlated, and results showed significant differences between administrators and practitioners on ratings of presence and organizational capacity. Implications for future research aimed at examining/building implementation practice capacity in community settings will be discussed.

© 2021 Wiley Periodicals LLC.

Keywords: behavioral health; capacity building; community-based organizations; evidence-based practices; implementation practice; implementation practice areas; implementation science

References

1. Aarons, G. A., Ehrhart, M. G., & Farahnak, L. R. (2014). The implementation scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science, 9(45), 45. https://doi.org/10.1186/1748-5908-9-45 - PubMed
2. Aarons, G. A., Hurlburt, M., & McCue Horwitz, S. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4-23. https://doi.org/10.1007/s10488-010-0327-7 - PubMed
3. Aarons, G. A., Sommerfeld, D. H., & Walrath-Greene, C. M. (2009a). Evidence-based practice implementation: the impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Implementation Science, 4(83), 83. https://doi.org/10.1186/1748-5908-4-83 - PubMed
4. Aarons, G. A., Wells, R. S., Zagursky, K., Fettes, D. L., & Palinkas, L. A. (2009b). Implementing evidence-based practice in community mental health agencies: A multiple stakeholder analysis. American Journal of Public Health, 99(11), 2087-2095. https://doi.org/10.2105/AJPH.2009.161711 - PubMed
5. Amaya-Jackson, L., Hagele, D., Sideris, J., Potter, D., Briggs, E. C., Keen, L., Murphy, R. A., Dorsey, S., Patchett, V., Ake, G. S., & Socolar, R. (2018). Pilot to policy: statewide dissemination and implementation of evidence-based treatment for traumatized youth. BMC Health Services Research, 18(598), 589. https://doi.org/10.1186/s12913-018-3395-0 - PubMed
6. Bach-Mortensen, A. M., Lange, B. C., & Montgomery, P. (2018). Barriers and facilitators to implementing evidence-based interventions among third sector organizations: A systematic review. Implementation Science, 13(103), 103. https://doi.org/10.1186/s13012-018-0789-7 - PubMed
7. Bauer, M. S., Damschroder, L., Hagedorn, H., Smith, J., & Kilbourne, A. M. (2015). An introduction to implementation science for the non-specialist. BMC Psychology, 3(32), 32. https://doi.org/10.1186/s40359-015-0089-9 - PubMed
8. Beehler, S. (2016). Ripple effects of implementing evidence-based mental health interventions in a community-based social service organization. Journal of Community Psychology, 44(8), 1070-1080. https://doi.org/10.1002/jcop.21820 - PubMed
9. Beidas, R. S., Steward, R. E., Adams, D. R., Fernandez, T., Lustbader, S., Powell, B. J., Aarons, G. A., Hoagwood, K. E., Evans, A. C., Hurford, M. O., Rubin, R., Hadley, T., Mandell, D. S., & Barg, F. K. (2016). A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 893-908. https://doi.org/10.1007/s10488-015-0705-2 - PubMed
10. Carlfjord, S., Roback, K., & Nilsen, P. (2017). Five years’ experience of an annual course on implementation science: An evaluation among course participants. Implementation Science, 12(1), 101. https://doi.org/10.1186/s13012-017-0618-4 - PubMed
11. Chinman, M., Hannah, G., Wandersman, A., Ebener, P., Hunter, S. B., Imm, P., & Sheldon, J. (2005). Developing a community science research agenda for building community capacity for effective preventive interventions. American Journal of Community Psychology, 35(3), 143-157. https://doi.org/10.1007/s10464-005-3390-6 - PubMed
12. Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage. - PubMed
13. Cusworth Walker, S., Sedlar, G., Berliner, L., Rodriguez, F. I., Davis, P. A., Johnson, S., & Leith, J. (2019). Advancing the state-level tracking of evidence-based practices: A case study. International Journal of Mental Health Systems, 13(25), 1-9. https://doi.org/10.1186/s13033-019-0280-0 - PubMed
14. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(50), 50. https://doi.org/10.1186/1748-5908-4-50 - PubMed
15. Dobni, C. B. (2008). Measuring innovation culture in organizations: The development of a generalized innovation culture construct using exploratory factor analysis. European Journal of Innovation Management, 11(4), 539-559. https://doi.org/10.1108/14601060810911156 - PubMed
16. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3-4), 327-350. https://doi.org/10.1007/s10464-008-9165-0 - PubMed
17. Edmunds, J. M., Beidas, R. S., & Kendell, P. C. (2013). Dissemination and implementation of evidence-based practices: Training and consultation as implementation strategies. Clinical Psychology: Science and Practice, 20(1), 152-165. https://doi.org/10.1111/cpsp.12031 - PubMed
18. Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ISC). Implementation Science, 9(157), 157. https://doi.org/10.1186/s13012-014-0157-1 - PubMed
19. Ehrhart, M. G., Torres, E. M., Green, A. E., Trott, E. M., Willging, C. E., Moullin, J. C., & Aarons, G. A. (2018). Leading for the long haul: A mixed-method evaluation of the Sustainment Leadership Scale (SLS). Implementation Science, 13(17), 17. https://doi.org/10.1186/s13012-018-0710-4 - PubMed
20. Fernandez, M. E., Walker, T. J., Weiner, B. J., Calo, W. A., Liang, S., Risendal, B., Friedman, D. B., Ping Tu, S., Williams, R. S., Jacobs, S., Herrmann, A. K., & Kegler, M. C. (2018). Developing measures to assess constructs from the inner setting domain of the Consolidated Framework for Implementation Research. Implementation Science, 13(52), 52. https://doi.org/10.1186/s13012-018-0736-7 - PubMed
21. Fixsen, D. L., Blase, K. A., & Van Dyke, M. (2019). Implementation practice and science. Active Implementation Research Network. - PubMed
22. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). https://nirn.fpg.unc.edu/sites/nirn.fpg.unc.edu/files/resources/NIRN-MonographFull-01-2005.pdf - PubMed
23. Frank, H. E., Becker-Haimes, E. M., & Kendall, P. C. (2019). Therapist training in evidence-based interventions for mental health: A systematic review of training approaches and outcomes. Clinical Psychology: Science and Practice, 27(3), 1-30. https://doi.org/10.1111/cpsp.12330 - PubMed
24. Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion intervention: The RE-AIM framework. American Journal of Public Health, 89(9), 1322-1327. https://doi.org/10.2105/ajph.89.9.1322 - PubMed
25. Glisson, C., & Schoenwald, S. K. (2005). The ARC organizational and community intervention strategy for implementing evidence-based children's mental health treatments. Mental Health Services Research, 7(4), 243-259. https://doi.org/10.1007/s11020-005-7456-1 - PubMed
26. Green, A. E., Albanese, B. J., Shapiro, N. M., & Aarons, G. A. (2014). The roles of individual and organizational factors in burnout among community-based mental health service providers. Psychological Services, 11(1), 41-49. https://doi.org/10.1037/a0035299 - PubMed
27. Langford, P. (2009). Measuring organizational climate and employee engagement: Evidence for a 7 Ps model of work practices and outcomes. Australian Journal of Psychology, 61(4), 185-198. https://doi.org/10.1080/00049530802579481 - PubMed
28. Last, B. S., Schriger, S. H., Timon, C. E., Frank, H. E., Buttenheim, A. M., Rudd, B. N., Fernandez-Marcote, S., Comeau, C., Shoyinka, S., & Beidas, R. S. (2021). Using behavioral insights to design implementation strategies in public mental health settings: A qualitative study of clinical decision-making. Implementation Science Communications, 2(6), 6. https://doi.org/10.1186/s43058-020-00105-6 - PubMed
29. Luke, D. A., Calhoun, A., Robichaux, C. B., Elliot, M. B., & Moreland-Russell, S. (2014). The program sustainability assessment tool: A new instrument for public health programs. Preventing Chronic Disease, 11, 130-184. https://doi.org/10.5888/pcd11.130184 - PubMed
30. Mandell, D. S. (2020). Traveling without a map: An incomplete history of the road to implementation science and where we may go from here. Administration and Policy in Mental Health and Mental Health Services Research, 47, 272-278. https://doi.org/10.1007/s10488-020-01013-6 - PubMed
31. Massey, O. T., & Vroom, E. B. (2020). The role of implementation science in behavioral health. In B. L. Levin, & A. Hanson (Eds.), Foundations of behavioral health (3rd ed.). Springer. - PubMed
32. Massey, O. T., Vroom, E. B., & Weston, A. (2021). Implementation of school-based behavioral health services over time: A longitudinal, multilevel study. School Mental Health, 13(1), 201-212. https://doi.org/10.1007/s12310-020-09407-5 - PubMed
33. Moore, J. E., Rashid, S., Park, J. S., Khan, S., & Straus, S. E. (2018). Longitudinal evaluation of a course to build core competencies in implementation practice. Implementation Science, 13(106), 106. https://doi.org/10.1186/s13012-018-0800-3 - PubMed
34. Newman, K., Van Eerd, D., Powell, B. J., Urquhart, R., Cornelissen, E., Chan, V., & Lal, S. (2015). Identifying priorities in knowledge translation from the perspective of trainees: results from an online survey. Implementation Science, 10(92), 92. https://doi.org/10.1186/s13012-015-0282-5 - PubMed
35. Park, J. S., Moore, J. E., Sayal, R., Holmes, B. J., Scarrow, G., Graham, I. D., Jeffs, L., Timmings, C., Rashid, S., Mascarenhas Johnson, A., & Straus, S. E. (2018). Evaluation of the “Foundations in Knowledge Translation” training initiative: preparing end users to practice KT. Implementation Science, 13(63), 63. https://doi.org/10.1186/s13012-018-0755-4 - PubMed
36. Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015). A refined compilation of implementation strategies: Results from the Exert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10(21), 21. https://doi.org/10.1186/s13012-015-0209-1 - PubMed
37. President's New Freedom Commission on Mental Health. (2003). Achieving the promise: Transforming mental health care in America. Final Report (DHHS Pub. No. SMA-03-3832). Department of Health and Human Services. https://govinfo.library.unt.edu/mentalhealthcommission/reports/FinalReport/downloads/FinalReport.pdf - PubMed
38. Proctor, E., Luke, D., Calhoun, A., McMillen, C., Brownson, R., McCrary, S., & Padek, M. (2015). Sustainability of evidence-based healthcare: Research agenda, methodological advances, and infrastructure support. Implementation Science, 10(88), 88. https://doi.org/10.1186/s13012-015-0274-5 - PubMed
39. Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 24-34. https://doi.org/10.1007/s10488-008-0197-4 - PubMed
40. Ramanadhan, S., Crisostomo, J., Alexander-Molloy, J., Gandelman, E., Grullon, M., Lora, V., Reeves, C., & Savage, C., PLANET MassCONECT C-PAC, & Viswanath, K. (2012). Perceptions of evidence-based programs among community-based organizations tackling health disparities: A qualitative study. Health Education Research, 27(4), 717-728. https://doi.org/10.1093/her/cyr088 - PubMed
41. Rogers, L., De Brun, A., Birken, S. A., Davies, C., & McAuliffe, E. (2020). The micropolitics of implementation: A qualitative study exploring the impact of power, authority, and influence when implementing change in healthcare teams. BMC Health Services Research, 20(1059), 1059. https://doi.org/10.1186/s12913-020-05905-z - PubMed
42. Schultes, M. T., Aijaz, M., Klug, J., & Fixsen, D. L. (2021). Competences for implementation science: What trainees need to learn and where they learn it. Advances in Health Sciences Education, 26, 19-35. https://doi.org/10.1007/s10459-020-09969-8 - PubMed
43. Stamatakis, K. A., McQueen, A., Filler, C., Boland, E., Dreisinger, M., Brownson, R. C., & Luke, D. A. (2012). Measurement properties of a novel survey to assess stages of organizational readiness for evidence-based interventions in community chronic disease prevention settings. Implementation Science, 7(65), 65. https://doi.org/10.1186/1748-5908-7-65 - PubMed
44. Stanhope, V., Choy-Brown, M., Barrenger, S., Manuel, J., Mercado, M., McKay, M., & Marcus, S. C. (2017). A comparison of how behavioral health organizations utilize training to prepare for health care reform. Implementation Science, 12(19), 19. https://doi.org/10.1186/s13012-017-0549-0 - PubMed
45. Tabak, R. G., Padek, M. M., Kerner, J. F., Stange, K. C., Proctor, E. K., Dobbins, M. J., Colditz, G. A., Chambers, D. A., & Brownson, R. C. (2017). Dissemination and implementation science training needs: Insights from practitioners and researchers. American Journal of Preventive Medicine, 52(3 Suppl. 3), S322-S329. https://doi.org/10.1016/j.amepre.2016.10.005 - PubMed
46. Ullrich, C., Mahler, C., Forstner, J., Szecsenyi, J., & Wensing, M. (2017). Teaching implementation science in a new Master of Science Program in Germany: A survey of stakeholder expectations. Implementation Science, 12(55), 55. https://doi.org/10.1186/s13012-017-0583-y - PubMed
47. Vroom, E. B., Massey, O. T., Martinez Tyson, D., Levin, B. L., & Green, A. L. (2021). Conceptualizing implementation practice capacity in community-based organizations delivering evidence-based behavioral health services. Global Implementation Research and Applications. https://doi.org/10.1007/s43477-021-00024-1 - PubMed
48. Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., Blachman, M., Dunville, R., & Saul, J. (2008). Bridging the gap between prevention research and practice: The Interactive Systems Framework for dissemination and implementation. American Journal of Community Psychology, 41(3-4), 171-181. https://doi.org/10.1007/s10464-008-9174-z - PubMed
49. Westerlund, A., Nilsen, P., & Sundberg, L. (2019). Implementation of implementation science knowledge: The research-practice gap paradox. Worldviews on Evidence-based Nursing, 16(5), 332-334. https://doi.org/10.1111/wvn.12403 - PubMed
50. Williams, N. J., & Beidas, R. S. (2019). Annual research review: The state of implementation science in child psychology and psychiatry: A review and suggestions to advance the field. Journal of Child Psychology and Psychiatry, 60(4), 430-450. https://doi.org/10.1111/jcpp.12960 - PubMed
51. Wilson, M. G., Lavis, J. N., & Guta, A. (2012). Community-based organizations in the health sector: A scoping review. Health Research Policy and Systems, 10(36), 1-9. https://doi.org/10.1186/1478-4505-10-36 - PubMed

Publication Types

• Journal Article

Grant support

• R25 DA035163/DA/NIDA NIH HHS/United States
• R25 DA050735/DA/NIDA NIH HHS/United States