If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Requests for reprints should be addressed to Wendy Levinson, MD, Department of Medicine, University of Toronto, Suite 3-805, R. Fraser Elliot Building, 190 Elizabeth Street, Toronto, ON M5G 2C4, Canada
A voluntary physician-led, nongovernmental process of setting standards is a unique strength of American medicine. Founded in 1936, the American Board of Internal Medicine (ABIM) has set the standards for practice in internal medicine and its subspecialties. ABIM is a physician-run organization but is independent of any physician societies or membership organizations. The standards, measured by the “certification” process, demonstrate to the public that internists have met the knowledge and practice requirements that ensure a high level of quality of care. It is a model for other countries that have standards set by either a government agency or a physician membership organization—which is often seen as less credible by the public.
Approximately 98% of internists who completed an accredited training program between 1990 and 2007 attempted certification; 96% ultimately achieved certification in internal medicine.
In 1990, the American Board of Internal Medicine (ABIM) made certification time limited: all certifications in internal medicine and its subspecialties would be valid for a period of 10 years, and internists would be required to maintain their certification through a Maintenance of Certification program.
ABIM plans to institute a more continuous process in keeping with external medical regulators and public expectations that internists will update their knowledge and skills regularly.
While participation in the certification process is entirely voluntary, 98% of internists who completed an accredited training program between 1990 and 2007 attempted certification. Of these individuals, 96% ultimately achieved certification in internal medicine. Until 1990, internists completing the ABIM standards were issued certifications that were valid indefinitely. However, there is growing evidence that physicians' knowledge and skills decline in the years after training,
ABIM instituted a significant change to the program in 1990: all certifications in internal medicine and its subspecialties would be valid for a period of 10 years, and internists would be required to maintain their certification through a Maintenance of Certification (MOC) program. Diplomates who did not maintain their certification would no longer be “board certified.” While the change to the program was and remains contentious
—with many internists objecting to these new and more periodic requirements—approximately 85% chose to participate in MOC. It is now 2 decades since MOC was introduced. This article describes the rationale, the components of the present program, the evolving research evidence, characteristics of internists who have participated over the last 20 years, and the limitations and future of MOC.
Rationale for MOC
There is significant evidence supporting the importance of board certification. Initial ABIM certification, which consists of training in an accredited program and passing the ABIM examination, is associated with the provision of higher-quality care by certified internists compared with physicians who are not certified. For example, studies have found that certified internists provide patients with higher rates of preventive services, improved care for hypertension, and lower mortality rates in patients with myocardial infarction compared with care delivered by noncertified internists.
However, certification is limited to assessing physicians at one point in time and does not ensure that they maintain their knowledge and skills in an environment of rapidly growing and expanding research evidence. Evidence suggests that physicians do not maintain their knowledge and skills or effectively self-assess their performance on their own.
conducted a systematic review of 62 articles about physician knowledge or a quality-of-care outcome and the years since graduation or the physician's age. Overall, 73% of the articles reported decreasing performance on some or all of the outcomes assessed with increasing years in practice. The studies assessed outcomes that are widely accepted as important to quality care, including general medical and specialized knowledge relevant to physicians' practice; adherence to standards of appropriate diagnostic, screening, or preventive care; adherence to standards of appropriate treatment; and actually measured patient outcomes. Furthermore, studies assessing how physicians actually perform in practice demonstrate major gaps between known appropriate standards of practice and what physicians actually do. Often, basic standards of care are performed in only 50% of patients.
that to provide the public with a credible standard, the profession must have a systematic way to assess physician knowledge, skills, and actual care delivery in a transparent and ongoing fashion.
MOC in 2010
ABIM is a member of the American Board of Medical Specialties (ABMS) – the umbrella organization for 26 specialty boards. ABMS articulates requirements for elements of the MOC program and allows the individual boards the flexibility to design their own programs to meet those requirements under the leadership of physicians in academia and practice. ABMS requires 4 components for every MOC program: evidence of professional standing (license to practice); participation in lifelong learning and self-assessment; evidence of cognitive expertise (examination); and assessment of practice performance.
The ABIM MOC program is designed to be responsive to new requirements from ABMS and feedback from the diplomates who participate in the program. Currently, the ABIM program requires diplomates to pass a secure examination in their subspecialty once every 10 years. Evidence supports that physicians should possess a certain level of knowledge and reasoning ability on their own to effectively make clinical decisions.
The examination is a computer-delivered test typically composed of 180 multiple-choice questions. The examination in each specialty is designed by a test-writing committee comprising specialists in that area; ABIM psychometricians ensure that each examination is reliable and valid.
Other elements of the ABIM MOC program allow physicians to choose either ABIM products or other products or programs offered by their society, health system, or hospital to meet those requirements. These non-ABIM programs are evaluated by ABIM physician-boards to ensure that they meet high standards for formative assessment. For example, while ABIM offers self-evaluation of medical knowledge assessments in internal medicine and each of its subspecialties, more than 30 products are ABIM-approved for credit towards self-evaluation of medical knowledge.
In response to the growing literature on the importance of self-assessment in practice and the idea that knowledge alone was not enough to determine if physicians had the skills and judgment to deliver quality care, ABIM added a self-assessment of practice performance requirement to the program in 2006. To help physicians meet this requirement, ABIM developed web-based tools—Practice Improvement Modules (PIMs) —that allowed physicians to examine elements of their practice and to get feedback from peers and patients. PIMs allow internists to assess how they are actually performing in their work setting and contain a clinical care audit with feedback, a survey of patient experience, and an assessment of the “systems” of a practice. Data collected by the physician or staff via the web are analyzed automatically by unique ABIM statistical software, which provides feedback about how a physician's performance compares with standards in the field. The internist then reflects on their performance and creates their own quality-improvement plan. At the end of the process, physicians resubmit data and can compare their performance before and after the quality-improvement intervention. This participation in a performance assessment and improvement cycle, often called a Plan-Do-Study-Act cycle, was developed out of the science of quality improvement.
Experience and assessment drive adult learning. Assessment helps physicians to recognize and address gaps in knowledge and performance. Although there are many models, Kolb's model of experiential learning (Figure 1) provides one example to highlight how assessment is a necessary element of learning and personal change.
In this model, assessment is a particularly important aspect of the concrete experience and observation and reflection phases. Physicians require “good data” to ensure that their interpretations and reflections are accurate around their performance and learning needs. However, research demonstrates that physicians are not accurate in generating this information through isolated self-assessment, and the least competent physicians are unfortunately also the least skilled in assessing their abilities and performance.
Furthermore, physicians often confuse experience with expertise. Evidence exists that overall competence may actually decline with age; older physicians may fail to “keep up” and are prone to “premature closure” when making a diagnosis.
It is not surprising, therefore, that these deficiencies may not get corrected in practice, especially if the physician does not participate in meaningful, ongoing, and rigorous assessment processes. MOC provides a useful assessment vehicle for practicing physicians to accurately and rigorously assess their knowledge and performance gaps.
What Is the Evidence for ABIM's MOC Program?
A small but evolving body of literature relates elements of MOC to physician performance. Further research is imperative to understand both the relationship of MOC to outcomes and the best assessment tools. The MOC secure examination is highly reliable, with coefficients consistently above 0.90.
Related to the PIMs, diplomates report that they value the information they receive about their practice, often describing the “aha” moment when they review their performance data and uncover an area of suboptimal practice where they are doing less well than expected.
In all, the 13 studies published since 2006 have evaluated PIMs in diabetes, preventive clinical services, preventive cardiology, care of the vulnerable elderly, asthma, communication with patients, communication with referring physicians, and the comprehensive care. Ongoing studies are exploring the impact of the PIM on patients, how to assess physician performance across multiple conditions, and the interaction between health system and physician competence.
Participants and Feedback
In 2010, approximately 70% of ABIM diplomates have at least one time-limited certificate with a 10-year duration. The other 30% completed their certification before the introduction of MOC and hold certification that is valid indefinitely; they are often called “grandfathers.” Several subspecialties, such as critical care and geriatrics, became subspecialties after 1987 and never had “grandfathers.” ABIM encourages grandfathers to recertify voluntarily,
Subspecialists can chose MOC for general internal medicine, their specialty, or both. Overall, the rates of completion of MOC in subspecialty is high (82%), but this rate varies from 50% in geriatrics to 89% in gastroenterology (Figure 2). In addition, a significant number of subspecialists also complete MOC in general internal medicine, ranging from 33% for cardiology to 86% for geriatrics. These high participation rates for a voluntary process indicate that internists and the organizations within which they practice place value on maintaining board-certification status.
ABIM asks diplomates to complete a survey about the value and experience regarding their participation on each component of MOC. For the self-assessment of knowledge modules, diplomates provided feedback for 85% of more than 74,000 modules completed between April 2007 and March 2009. Seventy-eight percent agreed or strongly agreed that the modules provided a valuable overall learning experience and helped them identify important areas for further study, while 71% said the knowledge modules raised their awareness on how to improve patient care.
Although the examination is one of the more anxiety-provoking aspects of the MOC program, diplomates rate the overall experience once they have taken the examination highly (84%), and most believe that their MOC examination is a relevant test of their knowledge (61%). Of the 23,646 diplomates who took an MOC examination between 2007 and 2009, 15% disagreed or strongly disagreed that their MOC examination was a fair assessment of clinical knowledge in their discipline, and 4% were dissatisfied with the testing experience.
Diplomates completed over 9800 PIMs between January 2008 and June 2009, with 79% of PIM completers providing feedback. For the disease- or condition-specific PIMs (eg, diabetes, hypertension), 90% of diplomates agreed or strongly agreed that reviewing their patient charts as part of the audit process raised their awareness of the quality of care their practice provided, while 88% said the audit provided ideas on how to improve their practice. Overall, 73% of diplomates reported that they changed their practice as a direct result of completing a PIM, and 82% reported they would recommend the PIM to a colleague.
Limitations and the Future of MOC
ABIM believes the MOC process must continue to improve and evolve. In a recent dialogue related to whether grandfathers should recertify, diplomates suggested areas of deficiency in the present MOC process and recommended changes.
Diplomates point out the need for the secure examination to be more tailored to their specific practice and provide electronic resources to look up information as one does in practice. They are concerned about the time required to complete the process and they call for better research to understand whether the time and effort are worth it in terms of patient outcomes. They raise concerns about the cost of the process.
The most important upcoming change is that the MOC program will become more continuous, requiring internists to participate on a regular basis. Currently, under the 10-year MOC program, the vast majority (74%) wait until the ninth year of their certification before they prepare any of the components of MOC. A more continuous process is consistent with public expectations that internists will update their knowledge and skills regularly,
and consistent with external medical regulators and medical organizations that require physicians to report their participation in continuing medical education on a yearly or biyearly basis. For example, most state licensing boards request information from all physicians about their continuing medical education biyearly to renew state licenses. Harmonizing MOC with these external stakeholders will ultimately make it easier for internists to participate in MOC and meet other requirements simultaneously. Furthermore, a continuous MOC process is aligned with educational theory that supports the idea that most effective learning is ongoing.
The MOC process should evaluate internists' competencies in a variety of domains. The ABMS competencies include patient care, medical knowledge, practice-based learning and improvement, systems-based learning, professionalism and interpersonal skills, and communication. Skill in working collaboratively with a team of health care providers is increasingly recognized as key for internists. However, the present MOC process emphasizes some competencies more than others; for example, professionalism and clinical skills are minimally evaluated. ABIM is developing new tools to help internists assess each of these competencies, but development of reliable and valid tools will require significant resources. ABIM recently launched efforts to develop useful assessment tools in teamwork and professionalism. In addition, ABMS now requires a component related to patient safety and ABIM is exploring options to incorporate it into MOC.
Specific elements are undergoing changes. The secure examination will be changed to improve fidelity, and research is under way to explore the inclusion of resources so that internists can look up information, simulating real practice. ABIM is evaluating other assessments of medical knowledge, including a tool that allows physicians to answer clinical questions generated at the point of care
and increased use of simulation in cardiology and other specialties. PIMs are being improved by incorporating more specific feedback about a diplomate's performance compared with national and regional benchmarks and by developing new standards. ABIM and the specialty societies are working on the self assessment of knowledge to increase products tailored to individual needs. Recognizing that supervision of and feedback to learners is an important component of the quality calculus, ABIM last year introduced the Clinical Supervision PIM for academic faculty that combines assessment of learner clinical skills with a focused chart audit of key safety measures and diagnostic reasoning. Already more than 450 faculty members have completed the Clinical Supervision PIM.
As MOC enters its second decade, it is clear that internists have effectively transitioned from a “point-in-time” certification to a maintenance program. The large majority of internists with time-limited certifications actively participate in the program, and newer internists expect to participate throughout their practice careers. As ABIM responds to the changing face of medicine and feedback from diplomates, the program must evolve and improve to remain accountable to the first and most important constituency, the American public.
Shared medical regulation in a time of increasing calls for accountability and transparency: comparison of recertification in the United States, Canada, and the United Kingdom.
Conflict of Interest: Dr. Levinson is a member of the Board of Directors of the American Board of Internal Medicine Foundation and Dr. Holmboe is Chief Medical Officer and Senior Vice President of the American Board of Internal Medicine. The authors declare no other conflicts of interest.
Authorship: This work is original. The authors meet the criteria for authorship and accept responsibility for the content of the manuscript.
I read with particular interest the article by Levinson and Holmboe,1 recently published in The American Journal of Medicine. These authors provide a very informative update on the American Board of Internal Medicine (ABIM) Maintenance of Certification (MOC) program.
I strongly disagree with the authors' statement that certification is voluntary.1 Because some hospitals require applicants for membership on their staffs to be certified or recertified and because there is a risk that managed care organizations may use certification as a criterion for admitting physicians to their panels or for prorating their fees, many physicians consider it virtually compulsory.