Changes to the American Board of Internal Medicine’s Maintenance of Certification Program: Preserving Core Assessment Values
Authors
-
American Board of Internal Medicine
Rebecca S. Lipner -
American Board of Internal Medicine Bradley G. Brossman
-
American Board of Internal Medicine Louis J. Grosso
Keywords:
Assessment, Core Values, Fairness, High Stakes Testing, Integrity, Maintenance of Certification, ValidityAbstract
Changes to the American Board of Internal Medicine’s Maintenance of Certification Program: Preserving Core Assessment Values
As Maintenance of Certification (MoC) for practicing physicians has evolved into more continuous programs, controversy has arisen over how physicians should self-regulate their profession and serve the public trust in doing so.
As a result, the American Board of Internal Medicine (ABIM) has focused on making substantial changes to its MoC program by inviting physician members of the internal medicine community to play critical roles in shaping the program’s processes through innovation. Much of the recent focus has been on changing the high-stakes assessment to one that is more relevant and authentic to practice, less burdensome and anxiety provoking for physicians, and more continuous over the course of their career. However, at the same time, ABIM has been careful to preserve the core values of integrity and fairness that are central to testing programs especially when making consequential decisions about test-takers in order to ensure that public expectations are met.
This paper lays out the case for a continuous certification program with consequences and the story of how substantial changes were made using an evidence-based approach involving the physician community.
Downloads
Metrics
Downloads
Published
How to Cite
Issue
Section
References
ABIM (2015, February 3). ABIM announces immediate changes to MoC program. Retrieved May 16, 2018, from www.abim. org/news/abim-announces-immediate-changes-to-mocprogram.aspx
ABIM (2015, September). Assessment 2020 Task Force Report. Retrieved May 16, 2018. www.abim.org/news/assessment2020-task-force-findings.aspx.
ABIM (2016, February 5). More physicians invited to rate exam topics by relevance in practice and to help set exam standard. Retrieved May 16, 2018, from www.abim.org/news/ abim-engages-physicians-on-updates-potential-changestomoc-assessments.aspx
ABIM (2016, April 11). New ABIM survey indicates physician interest in potential changes to MoC assessment. Retrieved September 13, 2018, from www.abim.org/news/ new-abim-survey-indicates-physician-interest-in-potentialchanges-to-moc-assessment.aspx
ABIM (2016, June 26). Score report video. Retrieved from www. youtube.com/watch?v=41tXCmn87Ik
ABIM (2018, February). Maintenance of Certification program completion rates. Retrieved May 16, 2018, from www.abim. org/~/media/ABIM%20Public/Files/pdf/statistics-data/ maintenance-of-certification-completion-rates.pdf ABIM (2018). Internal medicine: Maintenance of Certification (MoC) examination blueprint. Retrieved March 3, 2018, from www.abim.org/~/media/ABIM%20Public/Files/pdf/ exam-blueprints/maintenance-of-certification/internalmedicine. pdfcardio
ABIM (2018). MoC assessments in 2018. Retrieved May 16, 2018, from www.abim.org/maintenance-of-certification/ moc-faq/moc-assessments.aspx
AERA, APA, NCME (2014). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.
Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (pp. 514– 515). Washington, DC: American Council on Education.
Baron, R. J., & Braddock, C. H. I. (2016). Knowing what we don’t know — improving Maintenance of Certification. New England Journal of Medicine, 375(26), 2516–2517.
Baron, R. J., & Johnson, D. (2014). The American Board of Internal Medicine: evolving professional self-regulation. Annals of Internal Medicine, 161(3), 221–223.
Brossman, B. G. (2018). Score report design: a collaborative process between measurement, communications, and subject matter experts. Paper presented at the annual meeting of the National Council on Measurement in Education, New York, NY.
Bylsma, W., Arnold, G. K., Fortna, G. S., & Lipner, R. (2010). Where have all the general internists gone? Journal of General Internal Medicine, 25(10), 1020.
Centor, R. M., Fleming, D. A., & Moyer, D. V. (2014). Maintenance of Certification: beauty is in the eyes of the beholder. Annals of Internal Medicine,161(3), 226-227.
Cervero, R. M., & Gaines, J. K. (2015). The impact of CME on physician performance and patient health outcomes: an updated synthesis of systematic reviews. The Journal of Continuing Education in the Health Professions, 35(2), 131–138.
Chesluk, B., Eden, A., Hansen, E., Johnson, M., Reddy, S., Bernabeo, E., Gray, B. How physicians prepare for Maintenance-of-Certification exams: A qualitative study. Accepted Academic Medicine [In press].
Choudhry, N. K., Fletcher, R. H., & Soumerai, S. B. (2005). Systematic review: the relationship between clinical experience and quality of health care. Annals of Internal Medicine. 142(4), 260–273.
Cilliers, F. J., Schuwirth, L. W. T., Herman, N., Adendorff, H. J., & van der Vleuten, C. P. M. (2012). A model of the preassessment learning effects of summative assessment inmedical education. Advances in Health Sciences Education: Theory and Practice, 17(1), 39–53.
Davis, D. A., Mazmanian, P. E., Fordis, M., Harrison, R. V., Thorpe, K. E., & Perrier, L. (2006). Accuracy of physician self-assessment compared with observed measures of competence. JAMA: Journal of the American Medical Association, 296(9), 1094–1102.
Gierl, M. J., Lai, H., & Turner, S. R. (2012). Using automatic item generation to create multiple-choice test items. Medical Education, 46(8), 757–765.
Gray, B., Vandergrift, J., Landon, B., Reschovsky, J., & Lipner, R. (2018). Associations between American Board of Internal Medicine Maintenance of Certification status and performance on a set of Healthcare Effectiveness Data and Information Set process measures. Annals of Internal Medicine, 169(2), 97–105.
Gray, B., Vandergrift, J., & Lipner, R. (2018). Association between the American Board of Internal Medicine’s general internist’s Maintenance of Certification requirement and mammography screening for Medicare beneficiaries. Women’s Health Issues, 28(1), 35–41.
Gray, B., Vandergrift, J., Lipner, R. S., & Green, M. M. (2017). Comparison of content on the American Board of Internal Medicine Maintenance of Certification Examination with conditions seen in practice by general internists. JAMA, 317(22), 2317–2324.
Gray, B. M., Vandergrift, J. L., Johnston, M. M., Reschovsky, J. D., Lynn, L. A., Holmboe, E. S., McCullough, J.S., & Lipner, R. S. (2014). Association between imposition of a Maintenance of Certification requirement and ambulatory care-sensitive hospitalizations and health care costs. JAMA: Journal of the American Medical Association, 312(22), 2348.
HEDIS & Performance Measurement (2018). Retrieved July 4, 2018, from www.ncqa.org/hedis-quality-measurement
Holmboe, E. S., Wang, Y., Meehan, T. P., Tate, J. P., Ho, S.-y., Starkey, K. S., & Lipner, R. S. (2008). Association between Maintenance of Certification Examination scores and quality of care for Medicare beneficiaries. Archives of Internal Medicine, 168(13), 1396–1403.
Huynh, H. &J. C. Saunders (1980). “Accuracy of two procedures for estimating reliability of mastery tests. Journal of Educational Measurement, 17(4): 351–358.
Irons, M.B., & Nora, L.M. (2015). Maintenance of certification 2.0— strong start, continued evolution. New Engl J Med, 372, 104–106.
Kane, M. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality & Social Psychology, 77(6), 1121–1134.
Larsen, D. P., Butler, A. C., & Roediger III, H. L. (2009). Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial. Medical Education, 43(12), 1174–1181.
Lipner, R. S., Brossman, B. G., Samonte, K. M., & Durning, S. J. (2017). Effect of access to an electronic medical resource on performance characteristics of a certification examination. Annals of Internal Medicine, 167(5), 302–310.
Lipner, R. S., Bylsma, W. H., Arnold, G. K., Fortna, G. S., Tooker, J., & Cassel, C. K. (2006). Who is maintaining Certification in internal medicine--and why? A national survey 10 years after Initial Certification. Annals of Internal Medicine, 144(1), 29–36.
McDonald, F., Duhigg, L., Arnold, G., Hafer, R., & Lipner, R. (2018). The American Board of Internal Medicine Maintenance of Certification Examination and State Medical Board disciplinary actions: a population cohort study. Journal of General Internal Medicine (Preprints), 1–7.
Poniatowski, P. A., Dugosh, J. W., Baranowski, R. A., Arnold, G. K., Lipner, R. S., Dec, G., Jr., & Green, M. M. (2018). Incorporating physician input into a Maintenance of Certification Examination: A content validity tool [published online ahead of print March 26, 2019]. Acad Med.
Roediger, H. L., 3rd, & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27.
Smiley, W., & Clauser, J. (2019). Providing preliminary decisions using raw scores in an IRT scored exam. Paper presented at the 2019 American Educational Research Association, Toronto, ON, CA.
Teirstein, P. S., & Topol, E. J. (2015). The role of Maintenance of Certification programs in governance and professionalism.JAMA, 313(18), 1809–1810.