If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
The American Board of Internal Medicine (ABIM) has defined the “problem resident” as a learner who demonstrates problem behaviors significant enough to require intervention by program leadership, typically the residency program director or chief resident.
on the prevalence of “problem residents” in internal medicine residency training programs. Their survey of program directors reported performance problems in 6.9% of residents. We are unaware of any subsequent large studies in internal medicine of this important topic. Although the term “problem resident” has been used frequently, we will refer to these individuals as “residents in difficulty.”
The purpose of this study was to assess internal medicine program director experiences with residents in difficulty in the era of Accreditation Council for Graduate Medical Education (ACGME) competencies.
Internal medicine residents requiring remediation often have deficiencies in multiple competencies.
Deficiencies were identified in all competencies; remediation is most successful for medical knowledge (85.8%) and least successful for professionalism (41.2%).
Application materials rarely help program directors identify individuals at risk for difficulty during residency.
Performance deficiencies were rarely (5.6%) self-identified by residents.
Materials and Methods
The Association of Program Directors in Internal Medicine Survey Committee develops and deploys an annual survey of internal medicine residency program directors to address important issues in graduate medical education. The survey process has been described.
For this study, e-mail notifications with program-specific hyperlinks to a Web-based questionnaire were sent in August 2008 to each Association of Program Directors in Internal Medicine institutional member (program directors) (372), representing 97.1% of 383 US categoric internal medicine residencies.
Reminders were sent to nonresponders biweekly until the survey closed in November 2008.
The survey collected information on characteristics of the program and program directors in addition to the program director's experiences with residents in difficulty during the 2007-2008 academic year. The entire survey and results are available online.
The study was approved by the Mayo Clinic Institutional Review Board.
Program description, geographic region, ABIM 3-year rolling pass rate, government affiliation, accreditation cycle length, size, and program director tenure were compared among survey responders and nonresponders using Fisher exact tests or Welch's t tests. Logistic regression models were used to examine possible associations with program and program director characteristics for 2 outcomes: the proportion of residents in difficulty in a program and the ability of program directors with residents in difficulty to identify warning signs in application materials. All first-order interactions were assessed, and a conservative alpha level of 0.01 was used throughout. Models were fit using SAS statistical software (version 9.1; SAS Institute Inc, Cary, NC).
Of 372 program directors, 268 (72.0%) completed the survey. A comparison between responders and nonresponders revealed no significant differences (Table 1).
Table 1US Internal Medicine Residency Program Characteristics by 2008 Association of Program Directors in Internal Medicine Survey Response
A total of 197 programs (73.5%) reported having residents in difficulty. The point prevalence for the 2007-2008 academic year was 3.5% (532 residents of 15,031 filled positions). The mean number of residents in difficulty per program was 2.9 with a median of 2. The majority of program directors (52.6%) reported that the number of residents in difficulty was unchanged when compared with previous years, with the percentages indicating an increase (18.7%) or decrease (23.9%) being similar.
The majority of performance deficiencies (83.3%) in residents in difficulty were identified during inpatient rotations. Residents in difficulty also were identified in continuity clinics (32%), outpatient rotations (19.8%), and informal meetings (20.8%). Poor performance on the Internal Medicine In-Training Exam, end-of-rotation examinations, and US Medical Licensing Examination Step 3 identified residents in difficulty in 19.8%, 1.5%, and 1.0%, respectively. The most likely to identify residents in difficulty were faculty (69.5%), supervising residents (including chief residents) (62.9%), peers (52.8%), program directors (39.6%), and fellows (8.1%). Nurses (35.0%), nonclinical staff (14.7%), and nondepartment staff (8.1%) also identified residents in difficulty. Residents in difficulty infrequently were identified through patient complaints (8.1%) and adverse events (6.1%). Residents self-identified performance issues only 5.6% of the time.
Contributing factors identified in residents in difficulty included depression, anxiety, and personality disorders, which were associated with 32.6% of residents in difficulty reports. Learning disability was a factor in 6.6% of residents in difficulty, whereas illness, substance abuse, and divorce were reported less than 5% of the time.
Program directors reported multiple interventions to deal with residents in difficulty, most commonly informal discussion (52.3%). Formal remediation was reported in 38.2% of residents in difficulty. Disciplinary actions reported included formal warning (27.4%), “marginal” as defined by ABIM FasTrack
(13.7%), academic probation (12.6%), and dismissal (4.7%). Residents were required to repeat a rotation or an entire year in 15.4% and 3.9% of cases. A minority of residents, 7.9%, resigned from the programs.
Deficiencies in performance were noted in all ACGME competencies, most commonly in patient care (53.0%), followed by medical knowledge (47.9%). Deficiencies in organization/prioritization, communication, and professionalism also were observed frequently, each occurring in more than 40% of residents in difficulty (Figure 1) . The majority of program directors (76.9%) reported that residents in difficulty had deficiencies in multiple competencies.
Program directors reported varying success with remediation. Remediation was most successful for deficiencies in medical knowledge (85.9%) and least successful for problems with professionalism (48.6%) (Figure 1).
Only 68 program directors (34.5%) with residents in difficulty could retrospectively identify warning signs in applications that might have predicted poor performance. Factors considered somewhat or very helpful by program directors included poor performance on US Medical Licensing Examination (57.4%), interview (42.6%), resident input (38.2%), dean's letters (27.9%), and letters of recommendation (16.2%).
Community-based, university-affiliated programs were associated with a 62% increase in the odds of reporting residents in difficulty when compared with university-based programs (P=.001). Each 1% decrease in the ABIM 2006-2008 rolling pass rate was associated with a 2.6% increase in the odds of reporting residents in difficulty (P=.005). Programs with a program director annual salary of $250,000 or less were associated with a 2.25-fold increase in the odds of reporting residents in difficulty when compared with programs where the program director made more than $250,000 annually (P<.0001). Each additional year of tenure as a program director was associated with a 2.5% decrease in the odds of reporting residents in difficulty (P=.01).
The majority of programs have residents in difficulty. Differences in methodology prevent a direct comparison of prevalence with that previously reported.
The types of interventions required for remediation of residents in difficulty suggest that substantial resources are used for each resident. The persistence of residents in difficulty in most programs suggests that program directors will continue to need skills and resources for recognition and remediation of residents in difficulty.
Program directors reported deficiencies in all 6 ACGME competencies, as well as in organization and prioritization. An important finding is that the majority of residents in difficulty have deficiencies in multiple competencies. This finding also has been reported in family medicine
Of great importance, deficiencies in professionalism are relatively common in residents in difficulty (41.2%) but respond poorly to remediation. Likewise, poor outcomes remediating professionalism deficiencies have been reported in obstetric and gynecology
Program directors reported that residents rarely possess self-awareness to identify their own deficiencies. A systematic review of physicians (including learners) found a striking inability to assess their own performance, which supports our findings.
The survey allowed multiple responses to the question about the highest level of intervention needed for residents in difficulty. A total of 988 actions were reported for the 532 residents in difficulty, suggesting that more than 1 action was required for some. The most frequent remediation methods reported were feedback and mentoring. Surveys of program directors in surgery
have outlined methods of remediation and estimation of success with residents in difficulty.
Ideally, resident performance problems could be minimized through a more stringent selection of applicants. In this study, only one third of program directors thought that standard application materials were predictive of performance difficulties. A minority of program directors (15/268, 5.6%) reported a decrease in residents in difficulty because of improved selection processes. A systematic review found that academic success in medical school weakly predicted success as a resident.
less than one third of program directors reported it was somewhat or very helpful, even in retrospect, for identifying residents at risk for performance difficulties. The dean's letter or Medical Student Performance Evaluation may predict better performance when the student is ranked higher in comparative ratings.
Overall, we believe that simple screening strategies are unlikely to consistently and prospectively identify residents in difficulty.
Four program-related factors were associated with a lower prevalence of residents in difficulty: higher ABIM pass rates, university-based training, higher salaried program directors, and longer tenured program directors. It is not clear why these factors were associated with a decreased risk for residents in difficulty, but it certainly is an area for further investigation.
To our knowledge, this study is the first to report the relative success of remediation in specific ACGME competencies. One explanation for variable success of remediation may reflect limitations in assessment of specific competencies. For example, medical knowledge can be objectively measured and deficiencies can be addressed through assigned reading and study programs.
Other competencies, such as professionalism and systems-based practice, are more difficult to assess and reassess after intervention.
There are limitations to our study. Survey research is by definition observational and can therefore only assess associations rather than causality. There is intrinsic potential for recall bias; program directors could not report on residents with problems who did not come to their attention. The similarity of responders and nonresponders and the high response rate suggest that the measures of association are likely representative of the population of internal medicine residencies. Our survey results of internal medicine program directors demonstrate commonalities with other studies of this topic;
however, the findings may not be directly applicable to all disciplines. We cannot comment on how many residents required multiple interventions and which type of interventions were most successful for specific deficiencies. Future studies may help elucidate interventions most helpful for remediation of specific competency deficiencies.
Ten years after the landmark study by Yao and Wright,
residents in difficulty remain a concern for internal medicine program directors. Our results suggest that current application materials do not provide the information needed to avoid selecting future residents in difficulty. The majority of residents in difficulty have problems in multiple competencies. A comprehensive and multisource evaluation system is necessary to identify residents in difficulty, because individual residents are unable to see their own deficient behaviors. Remediation remains difficult for all the competencies, but most troubling is the challenge of remediating unprofessional behavior.
The authors thank the Association of Program Directors in Internal Medicine, the members of the Survey Committee, and the residency program directors who completed this survey. The authors also thank the Mayo Clinic Survey Research Center for assistance with survey design and data collection.
American Board of Internal Medicine. Chief Residents' Workshop on Problem Residents. Association of Program Directors in Internal Medicine (APDIM)
Funding: This study was supported in part by the Mayo Clinic Internal Medicine Residency Office of Educational Innovations as part of the Accreditation Council for Graduate Medical Education Educational Innovations Project.
Conflict of Interest: None.
Authorship: All authors had access to the data and played a role in writing this manuscript.
Short Communication presented at: the Association for Medical Education in Europe, August 30, 2011, Vienna, Austria.