Advertisement

Curricular Content of Internal Medicine Residency Programs: A Nationwide Report

      Perspectives Viewpoints
      • This is the first study to provide nationwide prevalence rates of competency training in Internal Medicine, with particular attention to the integrative competencies of system-based practice, practice-based learning and improvement, and communication skills.
      • Overall high rates of training are occurring in select topics for all 3 competencies.
      • There is room for improving training in nonhospital-based sites and sophisticated information technology use.
      Inadequate physician skills after the completion of residency training
      • Lynn L.
      • Hess B.J.
      • Weng W.
      • et al.
      Gaps in quality of diabetes care in internal medicine residency clinics suggest the need for better ambulatory care training.
      • Wiest F.C.
      • Ferris T.G.
      • Gokhale M.
      • et al.
      Preparedness of internal medicine and family practice residents for treating common conditions.
      • Blumenthal D.
      • Gokhale M.
      • Campbell E.G.
      • et al.
      Preparedness for clinical practice: reports of graduating residents at academic health centers.
      is a growing public concern. Calls for reform have been reiterated by multiple stakeholders.
      • Horwitz R.I.
      • Kassirer J.P.
      • Holmboe E.S.
      • et al.
      Internal medicine residency redesign: proposal of the Internal Medicine Working Group.
      • Holmboe E.S.
      • Bowen J.L.
      • Green M.
      • et al.
      Reforming internal medicine residency training. A report from the Society of General Internal Medicine's task force for residency reform.
      In response, the Accreditation Council for Graduate Medical Education (ACGME) has mandated that programs adopt new curricula requiring residents to learn the skills physicians need to practice in today's complex health care system. In particular, training program curricula are now organized around the 6 ACGME competencies (patient care, medical knowledge, professionalism, systems-based practice, practice-based learning, and communication skills) and program specific milestones to address some of the gaps in resident skill sets.

      Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Residency Education in Internal Medicine. Available at: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/2013-PR-FAQ-PIF/140_internal_medicine_07012013.pdf. Accessed October 7, 2014.

      Accreditation Council for Graduate Medical Education. Focus on the Future: Accreditation Council for Graduate Medical Education Annual Report 2011. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/ACGME-2011_AR_F.pdf. Accessed May 6, 2013.

      Despite the attention given to resident education and competence, little nationwide data describe the content and pedagogies used in Graduate Medical Education (GME). To our knowledge, only 1 prior study attempted to describe what residency programs nationwide are teaching. In 2008-2009, at the request of the Medicare Payment Advisory Commission (MedPAC), the RAND Corporation conducted a descriptive study of GME curricular content via semistructured interviews with a representative sample of internal medicine residencies. RAND concluded that the curricular content of residency programs had fallen short of expert recommendations. Areas of particular concern included the integrative competencies of systems-based practice, practice-based learning, and communication skills. In addition, RAND found deficiencies in the use of nonhospital-based training sites and use of information technology.

      Cordasco K, Horta M, Lurie N, et al. How are residency programs preparing our 21st century internists? A review of internal medicine residency programs' teaching on selected topics. Available at: http://www.RAND.org/pubs/working_papers/WR686.html. Accessed December 28, 2012.

      A major limitation of the RAND study involved its small sample size. Only 20 of 320 allopathic and 6 of 51 osteopathic internal medicine residencies (7%) were included. Although the RAND study authors considered those samples to be representative, they acknowledged that their results were not generalizable.

      Cordasco K, Horta M, Lurie N, et al. How are residency programs preparing our 21st century internists? A review of internal medicine residency programs' teaching on selected topics. Available at: http://www.RAND.org/pubs/working_papers/WR686.html. Accessed December 28, 2012.

      Nevertheless, the report had far-reaching effects: MedPAC made GME funding recommendations largely on the basis of the findings from the RAND report. More recently, the IOM has also made major GME funding recommendations.

      Medicare Payment Advisory Commission. Report to the Congress: improving incentives in the Medicare Program. Available at: http://www.medpac.gov/documents/Jun09_EntireReport. Accessed December 28, 2012.

      Medicare Payment Advisory Commission. Report to Congress: aligning incentives in Medicare. Available at: http://www.medpac.gov/documents/Jun10_EntireReport.pdf. Accessed December 28, 2012.

      Graduate Medical Education that meets the Nation's Health needs. IOM Report Available at: http://www.iom.edu/Reports/2014/Graduate-Medical-Education-That-Meets-the-Nations-Health-Needs.aspx. Accessed October 7, 2014.

      The aim of this study is to describe the curricular content of today's internal medicine residency programs and compare our findings with the RAND report. We limit our analysis to the 3 ACGME competencies that garnered the most public concern and were studied by RAND. We hope this article informs the GME community and policymakers about the progress made in teaching these integrative competencies.

      Materials and Methods

      The Association of Program Directors in Internal Medicine (APDIM) conducts a yearly survey of internal medicine residency directors in the United States. The survey contains a core section on basic program and program director demographics and a variable section on topics of current interest in internal medicine training. The details of survey development have been described.
      • Chaudhry S.I.
      • Khanijo S.
      • Halvorsen A.J.
      • et al.
      Accountability and transparency in graduate medical education expenditures.

      Alliance for Academic Internal Medicine. APDIM Survey Data. Available at: http://www.im.org/toolbox/surveys/apdimsurveydata/Pages/default.aspx?PF=1. Accessed December 28, 2012.

      In 2010, the APDIM Survey Committee aimed to ascertain the curricular content of allopathic internal medicine residency programs nationwide. Because we intended to compare results with those reported in the RAND study, we obtained permission to use verbatim questions from the RAND semistructured interviews and adapted them to a web-based structured survey. The methodology of the RAND study has been described in detail.

      Cordasco K, Horta M, Lurie N, et al. How are residency programs preparing our 21st century internists? A review of internal medicine residency programs' teaching on selected topics. Available at: http://www.RAND.org/pubs/working_papers/WR686.html. Accessed December 28, 2012.

      Briefly, RAND conducted semistructured interviews to a sample of 59 program directors randomly selected from online directories of the American College of Physicians and the American College of Osteopathic Internists. Twenty-six (44% of the random sample, 6.7% of all internal medicine program directors in the country) completed a phone interview. We included 53 of 55 questions asked by RAND, including 19 (of 21 total) on systems-based practice, 15 on practice-based learning, 11 on communication skills, 6 on care settings, and 2 on information technology. Finally, facilitators and barriers to teaching these topics were solicited as free-text responses. All 53 questions are publically accessible and can be found in the RAND report and on the APDIM survey website.

      Cordasco K, Horta M, Lurie N, et al. How are residency programs preparing our 21st century internists? A review of internal medicine residency programs' teaching on selected topics. Available at: http://www.RAND.org/pubs/working_papers/WR686.html. Accessed December 28, 2012.

      Alliance for Academic Internal Medicine. APDIM Survey Data. Available at: http://www.im.org/toolbox/surveys/apdimsurveydata/Pages/default.aspx?PF=1. Accessed December 28, 2012.

      Of note, the main objective of both the RAND and APDIM studies was to ascertain prevalence rates of teaching select topics by internal medicine residencies. It was beyond the scope of the RAND or APDIM studies to provide detailed descriptions of pedagogies, ascertain exactly why certain topics were or were not taught, or measure learning among residents. RAND asked program directors to self-report teaching quality using a 5-point scale; however, because of poor data quality, these questions were dropped from the final report. In August 2011, the APDIM e-mailed its structured web-based survey to its 365 member programs, representing 95.8% of the US categorical internal medicine residencies. Nonresponders were contacted in September and October. The survey was closed to responses in November 2011. This study was approved by the institutional review board of Mayo Clinic and administered by the Mayo Survey Research Center.

      Data Analysis

      We used SAS statistical software (version 9.1; SAS Institute, Inc, Cary, NC) for all statistical analyses. Data from publically available sources were combined with the APDIM survey data to categorize programs by type (university, community, community-based university-affiliated, military), size (number of ACGME-approved residency positions), American Board of Internal Medicine board pass rate, ACGME cycle length, and US Census Bureau location.

      American Medical Association. Graduate medical education: FREIDA online. Available at: https://freida.ama-assn.org/Freida/user/viewProgramSearch.do. Accessed February 12, 2013.

      Accreditation Council for Graduate Medical Education (ACGME). List of ACGME accredited programs and sponsoring institutions. Available at: http://www.acgme.org/adspublic. Accessed February 12, 2013.

      American Board of Internal Medicine. Residency program pass rates 2008-2010. Available at: http://www.abim.org/pdf/pass-rates/residency-program-pass-rates.pdf. Accessed February 12, 2013.

      US Census Bureau. Census regions and divisions of the United States. Available at: http://www.census.gov/geo/www/us_regdiv.pdf. Accessed February 12, 2013.

      Responders were compared with nonresponders using Fisher exact tests and t tests as appropriate.
      Only 20 of the 26 programs surveyed by RAND in 2008-2009 were allopathic residencies that might have been surveyed again by the APDIM in 2011-2012. Because these 20 programs constituted such a small proportion of the APDIM sample (5%), all 20 programs were kept in our data analysis. The RAND study used semistructured telephone interviews and reported results as proportions. The APDIM survey questions were taken verbatim from RAND but structured in nature. Thus, we also calculated proportions and frequencies of teaching topics in systems-based practice, practice-based learning, and communication skills, as well as use of care settings and information technology.
      We compared the proportion of respondents answering affirmatively in the RAND and APDIM survey corresponding items using Fisher exact tests of proportions. Because of multiple comparisons, we used P < .01 for significance. If programs reported teaching a particular curricular item, we summarized self-reported teaching quality using the same 5-point Likert scale used by RAND investigators (5 = excellent; 1 = poor).
      One author (CL) coded all free-text comments on facilitators and barriers into the same 6 general categories used by RAND in their study, namely, accessibility and incorporation of information technology infrastructure; availability or lack of faculty resources and time; characteristics of the program, such as its setting and resources; degree of institutional support; level of residents' baseline knowledge, interest, and time; and scarcity of evidence on optimal methods for teaching the competencies. A second author (JE) coded a 10% sample (ie, every 10th free-text comment) of all comments made. A Kappa statistic was ascertained to assess reliability of coding between the 2 authors.

      Results

      The response rate was 72% (261/365) (Table 1). There was no statistical difference between responders and nonresponders in terms of program type, government affiliation, size of program, 3-year American Board of Internal Medicine board pass rate, ACGME cycle length, or years of experience of the program director. Compared with nonresponders, responders were more likely to be located in the West than in other US Census Bureau locations of the country (P < .007). Tables 2 and 3 show the proportion of programs self-reporting teaching topics in the APDIM and RAND studies. Table 4 notes facilitators and barriers. Figure 1 depicts data from Tables 2 and 3 as a box plot of composite affirmative responses to teaching as reported by RAND in 2008-2009 versus reported by APDIM in 2011-2012.
      Table 1Demographic Characteristics of Internal Medicine Residency Programs
      Program CharacteristicsResponders (n = 261)Nonresponders (n = 120)P Value
      QualitativeNo. (%)No. (%)
      Description (FREIDA).61
       Community-based29 (11)17 (14)
       Community-based, university affiliated135 (52)61 (51)
       University-based92 (35)38 (32)
       Military-based5 (2)4 (3)
      Census Region (US Census Bureau)
       Midwest61 (23)28 (23).007
       Northeast87 (33)46 (38)
       West44 (17)8 (7)
       South67 (26)32 (27)
       Other2 (1)6 (5)
      Government Affiliation? (ACGME)
       No198 (76)89 (74).69
      QuantitativeMean (SD)Mean (SD)P Value
      Program Size (No. ACGME positions)66 (38)63 (39).55
      ABIM pass rate 2008-2010, %89 (8.5)90 (7.5).36
      ACGME cycle length4.9 (1.4)4.8 (1.6).66
      Years working as Program Director7.3 (6.4)6.3 (6.7).16
      ABIM = American Board of Internal Medicine; ACGME = Accreditation Council for Graduate Medical Education; SD = standard deviation.
      Table 2Reported Prevalence of Teaching Systems-based Practice/Practice-based Learning
      APDIM Survey (n = 261)APDIM Survey (n = 261)
      Systems Based PracticeYes No. (%)
      Percentage is derived by using the total number of “yes” responses to the question “Does your program engage in or provide this activity?” as the numerator and the total number of responses excluding blank responses as the denominator (range of denominator n = 237 to n = 256).
      Excellent and Very Good No. (%)
      Percentage of responses derived by using the total number of “excellent” and “very good” responses to the question “The training your program provides in this subject is” as the numerator and the total number of responses excluding blank responses as the denominator (range of denominator n = 70 to n = 249).
      Patient Safety Issues and Methods
       1 Lectures/activity on patient safety issues248 (98)103 (41)
       2 Lectures on critical incidents226 (89)84 (38)
       3 Patient safety project182 (72)75 (40)
      Coordination of Care
       4 Formal instruction in inpatient provider handoffs234 (91)92 (40)
       5 Faculty/chief resident supervise sign-outs167 (66)72 (41)
       6 Inpatient written sign-outs critiqued151 (60)44 (26)
       7 Computer-based tool for sign-outs203 (80)83 (41)
       8 Formal instruction in discharge coordination192 (76)77 (38)
       9 IT supports discharge communication/coordination181 (72)58 (32)
       10 Formal instruction in outpatient coordination151 (60)38 (24)
       11 IT supports outpatient coordination134 (53)32 (25)
       12 Outpatient. IT enables sending of patient-care tasks117 (48)36 (31)
      Multidisciplinary Teams
       13 Formal multidisciplinary inpatient teams210 (82)116 (55)
       14 Semi-formal multidisciplinary inpatient teams only137 (58)63 (34)
       15 Formal multidisciplinary outpatient teams89 (35)34 (33)
       16 Semi-formal multidisciplinary outpatient teams141 (58)37 (23)
      Awareness of Absolute and Relative Cost
       17 Lectures on costs/cost-effectiveness201 (79)38 (19)
       18 Lectures on patient share of costs73 (29)12 (15)
       19 Other areas training in systems-based practice101 (43)n/a
      Practice Based Learning
      Quality Improvement Methods
       1 Lectures/computer-based training in QA219 (89)97 (46)
       2 Hospital administration collects/analyzes/presents data in quality measures230 (93)95 (43)
       3 Each resident works on quality assessment200 (82)93 (49)
       4 Residents collect or are provided data on their own patients209 (85)89 (43)
       5 Use chronic disease registries114 (46)40 (36)
       6 Lectures/computer-based training in implementing system change163 (67)60 (37)
       7 Each resident does project implementing systems change128 (58)51 (40)
      Evidence-based Medicine
       8 Formal session, searching literature223 (91)26 (54)
       9 Journal Club/EBM Conference243 (100)30 (64)
       10 Lectures on critiquing literature228 (93)n/a
       11 EBM assignments209 (85)n/a
      Clinical Decision Aids
       12 Use ordersets or critical pathways238 (97)n/a
       13 Formal lectures on clinical prediction143 (58)135 (40)
       14 IT supports clinical prediction tools121 (49)138 (42)
       15 Other areas, training in practice-based learning and improvement49 (23)n/a
      APDIM = Association of Program Directors in Internal Medicine; EBM = evidence-based medicine; IT = information technology; QA = quality assurance.
      Percentage is derived by using the total number of “yes” responses to the question “Does your program engage in or provide this activity?” as the numerator and the total number of responses excluding blank responses as the denominator (range of denominator n = 237 to n = 256).
      Percentage of responses derived by using the total number of “excellent” and “very good” responses to the question “The training your program provides in this subject is” as the numerator and the total number of responses excluding blank responses as the denominator (range of denominator n = 70 to n = 249).
      Table 3Reported Prevalence of Teaching Topics in Interpersonal and Communication Skills, Care Setting, and Information Technology
      Interpersonal and Communication SkillsAPDIM Survey (n = 261)APDIM Survey (n = 261)
      No. (%)
      Percentage is derived by using the total number of “yes” responses to the question “Does your program engage in or provide this activity?” as the numerator and the total number of responses excluding blank responses as the denominator (range of denominator n = 243 to n = 248).
      Excellent and Very Good

      No. (%)
      Percentage of responses derived by using the total number of “excellent” and “very good” responses to the question “The training your program provides in this subject is” as the numerator and the total number of responses excluding blank responses as the denominator (range of denominator n = 188 to n = 238).
       1 Communicating with other healthcare providers235 (95)120 (53)
       2 Communicating clearly with patients regarding diagnosis and treatment plans236 (96)117 (48)
       3 Engaging patients in shared decision making220 (90)84 (38)
       4 Providing patient education about self-care activities207 (83)60 (29)
       5 Providing counseling to enhance adherence or behavior change218 (89)68 (31)
       6 Communicating with patients with low health literacy193 (78)45 (23)
       7 Acquiring cultural competency224 (91)65 (29)
       8 Working with interpreters205 (84)70 (34)
       9 Communicating with patients regarding end-of-life decisions or palliative care243 (100)141 (59)
       10 Communicating with patient's family/significant others, ie, family meetings232 (95)118 (51)
       11 Other areas, training in interpersonal and communication skills86 (38)n/a
      Care Settings and Information Technology
       1 Medical home care setting training116 (46)26 (23)
       2 Managed care setting training140 (56)30 (20)
       3 Required experience in community clinic/private office setting189 (72)n/a
       4 Required experience providing hospice/palliative care179 (69)n/a
       5 Required experience in conducting home visits172 (66)n/a
       6 Required experience in nursing homes/rehabilitation units96 (37)n/a
       7 Using electronic medical records224 (92)135 (61)
       8 Using computer order entry196 (80)138 (71)
      APDIM = Association of Program Directors in Internal Medicine; n/a = not available.
      Percentage is derived by using the total number of “yes” responses to the question “Does your program engage in or provide this activity?” as the numerator and the total number of responses excluding blank responses as the denominator (range of denominator n = 243 to n = 248).
      Percentage of responses derived by using the total number of “excellent” and “very good” responses to the question “The training your program provides in this subject is” as the numerator and the total number of responses excluding blank responses as the denominator (range of denominator n = 188 to n = 238).
      Table 4Facilitators and Barriers to Teaching Systems-based Practice, Practice-based Learning, and Interpersonal and Communication Skills
      Systems-based Practice Comments
      Facilitator Comments (n = 163)Barrier Comments (n = 124)
      No. (%)No. (%)
      Program setting38 (23)14 (11)
      Faculty expertise and time42 (26)27 (22)
      Institutional support52 (32)20 (16)
      Resident time, knowledge, interest7 (4)30 (24)
      Research in education and evaluation17 (10)7 (6)
      IT6 (4)14 (11)
      Miscellaneous1 (2)12 (10)
      Practice-based Learning Comments
      Facilitator Comments (n = 123)Barrier Comments (n = 73)
      No. (%)No. (%)
      Program setting16 (13)4 (5)
      Faculty expertise and time41 (33)19 (26)
      Institutional support26 (21)6 (8)
      Resident time, knowledge, interest8 (7)10 (14)
      Research in education and evaluation14 (11)6 (8)
      IT16 (13)22 (30)
      Miscellaneous2 (2)6 (8)
      Interpersonal and Communication Skills Comments
      Facilitator Comments (n = 158)Barrier Comments (n = 102)
      No. (%)No. (%)
      Program setting39 (24)20 (20)
      Faculty expertise and time57 (36)32 (31)
      Institutional support19 (12)10 (10)
      Resident time, knowledge, interest7 (4)24 (24)
      Research in education and evaluation35 (22)7 (7)
      IT1 (1)1 (1)
      Miscellaneous0 (0)8 (8)
      IT = information technology.
      Figure thumbnail gr1
      Figure 1Affirmative responses to teaching in core competencies. APDIM = Association of Program Directors in Internal Medicine; IP & CS = interpersonal and communication skills; IT = information technology.

      Systems-Based Practice

      This competency was assessed with 19 survey items, including the domains of patient safety, coordination of care, multidisciplinary teams, and cost awareness. In 10 of the 19 survey items, the APDIM study found at least 30% higher prevalence of instruction than RAND (P < .001). Similar rates of teaching were seen in patient safety lectures, information technology–supported discharge communication, outpatient coordination, training about multidisciplinary teams, and teaching residents about patient share of costs. For 14 of the 19 curricular items, one quarter of programs reported their teaching to be very good or excellent.

      Practice-Based Learning and Improvement

      This competency was assessed with 15 survey items, including the domains of quality improvement, evidence-based medicine, and clinical decision aids. In 9 of the 15 survey questions, the APDIM study found at least 30% higher prevalence of instruction than RAND (P < .001). Similar rates were seen for quality assessment work, use of chronic disease registries, searching literature, and clinical prediction rules. For all 15 items, at least one quarter of all programs reported teaching to be very good or excellent.

      Interpersonal and Communication Skills

      This competency included 11 survey items. In 6 of the 10 items, the APDIM study found at least 30% higher prevalence of instruction than RAND (P < .001). Similar rates were reported for communicating about diagnosis/treatment and cultural competency. For all 11 items, one quarter of programs reported teaching to be very good or excellent.

      Clinical Care Settings and Information Technology

      A total of 8 items assessed care settings and the use of information technology. Training in nursing homes was lower, and training in medical homes was higher than reported by RAND (P < .001). All other items in this category (managed care settings, private practice settings, hospice settings, and home visits) were nonsignificant between the RAND and APDIM studies.
      The majority of programs are using electronic medical records (92%) and computer order entry (80%), although fewer are using information technology in more complex and sophisticated ways, such as in clinical prediction (49%), chronic disease registries (46%), or outpatient coordination of tasks (53%). Other than a borderline significant increase in computer order entry (P = .014), all other information technology items remained nonsignificant compared with the RAND results.

      Facilitators and Barriers

      Respondents provided a total of 743 free-text responses noting facilitators and barriers. A 10% sample (ie, every 10th response sampled) of 74 free-text responses was analyzed for agreement by a second author. The authors excluded comments that did not fit into any of the RAND codes (1 excluded by CL and 2 by JE). The kappa for agreement between CL and JE was moderate at 0.6 (standard error, 0.06).
      Faculty expertise was the most common facilitator for teaching practice-based learning and communication skills, whereas institutional support was noted for systems-based practice. Resident factors, information technology issues, and faculty expertise were the most frequently cited barriers to teaching for systems-based practice, practice-based learning, and communication skills, respectively.

      Discussion

      In its 2009 and 2010 reports to Congress, MedPAC made broad recommendations to Congress regarding the amount and distribution of GME funds. Much of MedPAC's concern centered on the perceived inadequacy of teaching the integrative competencies.

      Medicare Payment Advisory Commission. Report to the Congress: improving incentives in the Medicare Program. Available at: http://www.medpac.gov/documents/Jun09_EntireReport. Accessed December 28, 2012.

      Medicare Payment Advisory Commission. Report to Congress: aligning incentives in Medicare. Available at: http://www.medpac.gov/documents/Jun10_EntireReport.pdf. Accessed December 28, 2012.

      However, little nationwide data support or refute these concerns. Small single-site studies have emerged in the pediatric, orthopedic, ophthalmologic, and general surgery literature commenting on teaching select competencies.
      • Kesselheim J.C.
      • Sectish T.C.
      • Joffe S.
      Education in professionalism: results from a survey of pediatric residency program directors.
      • Roberts S.M.
      • Jarvis-Selinger S.
      • Pratt D.D.
      • et al.
      Reshaping orthopedic resident education in systems-based practice.
      • Lee A.G.
      • Oetting T.
      • Beaver H.A.
      • Carter K.
      The ACGME Outcome Project in ophthalmology: practical recommendations for overcoming the barriers to local implementation of the national mandate.
      • Greenberg J.A.
      • Irani J.L.
      • Greenberg C.C.
      • et al.
      The ACGME competencies in the operating room.
      To our knowledge, ours is the first study to provide an in-depth nationwide prevalence of competency training in internal medicine. Because internal medicine represents the largest constituency of GME (20%),

      Accreditation Council for Graduate Medical Education (ACGME). List of ACGME accredited programs and sponsoring institutions. Available at: http://www.acgme.org/adspublic. Accessed February 12, 2013.

      our study holds particular importance as policymakers grapple with leveraging the approximately $10 billion Medicare spends annually on GME dollars.

      Graduate Medical Education that meets the Nation's Health needs. IOM Report Available at: http://www.iom.edu/Reports/2014/Graduate-Medical-Education-That-Meets-the-Nations-Health-Needs.aspx. Accessed October 7, 2014.

      • Chaudhry S.I.
      • Khanijo S.
      • Halvorsen A.J.
      • et al.
      Accountability and transparency in graduate medical education expenditures.
      Our study highlights important changes made by residency programs in teaching the integrative competencies in the past several years. The results represent a significant rate of increase by training programs toward achieving the educational goals identified in the RAND study. Because our sample size was large, we provide a more accurate representation of the prevalence of integrative competency training than achieved by RAND. We identify significant increases in teaching of systems-based practice, practice-based learning, and communication skills, all competencies encompassing the skills needed by physicians entering today's practice environment. Lesser progress has been made in the areas of outpatient care and use of nonhospital-based training, possibly because of challenges around GME reimbursement, higher costs involved in outpatient training, and hospital reliance on the resident work force.
      • Brotherton S.E.
      • Etzel S.I.
      Graduate Medical Education, 2011-2012.
      • Nasca T.J.
      • Veloski J.J.
      • Monnier J.A.
      • et al.
      Minimum instructional and program-specific administrative costs of educating residents in internal medicine.
      Our results call for enhanced support in areas of training that have not improved since 2008.

      Differences

      We identify 3 main reasons our findings differ from the RAND study. Our study was conducted 3 years later (academic year 2011-2012 vs academic year 2008-2009). During this time, specifically in July 2009, the ACGME revised its program rules for internal medicine. The improvements seen in the curricular content of the integrative competencies and the use of care settings and information technology may be due to an ACGME mandate in these competencies. Comments used to assess facilitators revealed that program directors thought their institutions directed resources toward educational initiatives that were mandated by accreditation standards.
      Our study included 96% of all allopathic internal medicine residencies and achieved a 72% response rate, whereas the RAND study randomly sampled 7% of osteopathic and allopathic programs and achieved a 44% response rate.
      Although we carefully replicated the questions of the RAND study, theirs used semistructured telephone interviews and ours was a web-based structured survey. During semistructured telephone interviews, the RAND study followed each “yes” answer with prompts for the program director to further describe the relevant aspects of their program. Anticipation of these follow-up questions may have influenced program director responses.

      Study Strengths

      The strengths of our study include its large sample size and response rate, which helps ameliorate sampling and response bias. We believe our study represents the only comprehensive generalizable data on what internal medicine programs are teaching in the integrative competencies, what care settings they are using, and how information technology is used in their programs. While further work is needed to confirm these results and expand available knowledge into other specialties, this is a solid starting point to establishing meaningful, real world metrics to enhance residency training. Second, we used verbatim questions from the RAND study to allow easy and direct comparisons between the 2 sets of results. Finally, we study an issue that is of timely concern given the tenuous political climate that exists in GME funding.

      Graduate Medical Education that meets the Nation's Health needs. IOM Report Available at: http://www.iom.edu/Reports/2014/Graduate-Medical-Education-That-Meets-the-Nations-Health-Needs.aspx. Accessed October 7, 2014.

      • Brotherton S.E.
      • Etzel S.I.
      Graduate Medical Education, 2011-2012.
      Given the recent report by the Institute of Medicine's Committee on the Governance and Financing of Graduate Medical Education that calls for the inclusion of performance-based metrics in the mechanism of GME funding, we believe our study can inform that discussion and help focus efforts on those areas that are in greatest need of improvement and most beneficial to society. While further work is needed to confirm these results and expand available knowledge into other specialties, this is a solid starting point to establishing meaningful, real world metrics to enhance residency training.

      Study Limitations

      First, like any survey, our study may suffer from self-report bias. True prevalence rates of teaching systems-based practice, practice-based learning, and communication skills may be lower than those reported in this article. Second, we studied only internal medicine programs. However, internal medicine does constitute the largest proportion of all GME training positions nationwide. Also, it was our intent to study changes in teaching the integrative competencies over time and in comparison with the RAND study of internal medicine programs. Third, we did not ask respondents about the other 3 ACGME competencies, namely, patient care, medical knowledge, and professionalism. Finally, although we asked about facilitators and barriers, detailed information about the quality of teaching, pedagogies explored, resident learning, or other details was beyond the scope of this study.

      Conclusions

      Our study illustrates several points for policymakers to consider. Although there is room for improvement, programs are making significant strides toward teaching the skills today's physicians need on entering practice. Outpatient care and use of nonhospital-based training need attention. Whereas general information technology use is high, more sophisticated use, such as information technology to enhance coordination of care, use of disease registries, and critical pathways, needs support, especially as the nation moves toward meaningful use. We hope the findings in this report are useful to policymakers charged with allocating resources in a manner that ensures a healthy GME system capable of producing physicians well suited for 21st century health care.

      Acknowledgments

      The authors thank the APDIM for support, the members of the APDIM Survey Committee, and the residency program directors who completed the APDIM survey. Although we are reporting the results of the APDIM Survey, we are not presuming to speak for the organization, and our article does not constitute an official policy statement of the APDIM, the Alliance for Academic Internal Medicine, or any other organization with which any of the authors may be affiliated. The authors also thank the Mayo Clinic Survey Research Center for assistance with the survey design and data collection and the RAND Corporation for the use of study questions and collaboration on this article.

      References

        • Lynn L.
        • Hess B.J.
        • Weng W.
        • et al.
        Gaps in quality of diabetes care in internal medicine residency clinics suggest the need for better ambulatory care training.
        Health Aff (Millwood). 2012; 31: 150-158
        • Wiest F.C.
        • Ferris T.G.
        • Gokhale M.
        • et al.
        Preparedness of internal medicine and family practice residents for treating common conditions.
        JAMA. 2002; 288: 2609-2614
        • Blumenthal D.
        • Gokhale M.
        • Campbell E.G.
        • et al.
        Preparedness for clinical practice: reports of graduating residents at academic health centers.
        JAMA. 2001; 286: 1027-1034
        • Horwitz R.I.
        • Kassirer J.P.
        • Holmboe E.S.
        • et al.
        Internal medicine residency redesign: proposal of the Internal Medicine Working Group.
        Am J Med. 2011; 124: 806-812
        • Holmboe E.S.
        • Bowen J.L.
        • Green M.
        • et al.
        Reforming internal medicine residency training. A report from the Society of General Internal Medicine's task force for residency reform.
        J Gen Intern Med. 2005; 20: 1165-1172
      1. Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Residency Education in Internal Medicine. Available at: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/2013-PR-FAQ-PIF/140_internal_medicine_07012013.pdf. Accessed October 7, 2014.

      2. Accreditation Council for Graduate Medical Education. Focus on the Future: Accreditation Council for Graduate Medical Education Annual Report 2011. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/ACGME-2011_AR_F.pdf. Accessed May 6, 2013.

      3. Cordasco K, Horta M, Lurie N, et al. How are residency programs preparing our 21st century internists? A review of internal medicine residency programs' teaching on selected topics. Available at: http://www.RAND.org/pubs/working_papers/WR686.html. Accessed December 28, 2012.

      4. Medicare Payment Advisory Commission. Report to the Congress: improving incentives in the Medicare Program. Available at: http://www.medpac.gov/documents/Jun09_EntireReport. Accessed December 28, 2012.

      5. Medicare Payment Advisory Commission. Report to Congress: aligning incentives in Medicare. Available at: http://www.medpac.gov/documents/Jun10_EntireReport.pdf. Accessed December 28, 2012.

      6. Graduate Medical Education that meets the Nation's Health needs. IOM Report Available at: http://www.iom.edu/Reports/2014/Graduate-Medical-Education-That-Meets-the-Nations-Health-Needs.aspx. Accessed October 7, 2014.

        • Chaudhry S.I.
        • Khanijo S.
        • Halvorsen A.J.
        • et al.
        Accountability and transparency in graduate medical education expenditures.
        Am J Med. 2012; 125: 517-522
      7. Alliance for Academic Internal Medicine. APDIM Survey Data. Available at: http://www.im.org/toolbox/surveys/apdimsurveydata/Pages/default.aspx?PF=1. Accessed December 28, 2012.

      8. American Medical Association. Graduate medical education: FREIDA online. Available at: https://freida.ama-assn.org/Freida/user/viewProgramSearch.do. Accessed February 12, 2013.

      9. Accreditation Council for Graduate Medical Education (ACGME). List of ACGME accredited programs and sponsoring institutions. Available at: http://www.acgme.org/adspublic. Accessed February 12, 2013.

      10. American Board of Internal Medicine. Residency program pass rates 2008-2010. Available at: http://www.abim.org/pdf/pass-rates/residency-program-pass-rates.pdf. Accessed February 12, 2013.

      11. US Census Bureau. Census regions and divisions of the United States. Available at: http://www.census.gov/geo/www/us_regdiv.pdf. Accessed February 12, 2013.

        • Kesselheim J.C.
        • Sectish T.C.
        • Joffe S.
        Education in professionalism: results from a survey of pediatric residency program directors.
        J Grad Med Educ. 2012; 4: 101-105
        • Roberts S.M.
        • Jarvis-Selinger S.
        • Pratt D.D.
        • et al.
        Reshaping orthopedic resident education in systems-based practice.
        J Bone Joint Surg Am. 2012; 94: e1131-e1137
        • Lee A.G.
        • Oetting T.
        • Beaver H.A.
        • Carter K.
        The ACGME Outcome Project in ophthalmology: practical recommendations for overcoming the barriers to local implementation of the national mandate.
        Surv Ophthalmol. 2009; 54: 507-517
        • Greenberg J.A.
        • Irani J.L.
        • Greenberg C.C.
        • et al.
        The ACGME competencies in the operating room.
        Surgery. 2007; 142: 180-184
        • Brotherton S.E.
        • Etzel S.I.
        Graduate Medical Education, 2011-2012.
        JAMA. 2012; 308: 2264-2279
        • Nasca T.J.
        • Veloski J.J.
        • Monnier J.A.
        • et al.
        Minimum instructional and program-specific administrative costs of educating residents in internal medicine.
        Arch Intern Med. 2001; 161: 760-766