Advertisement

Milestones for Internal Medicine Sub-interns

      Abstract

      Background

      As residency programs move toward measuring milestones for competency-based education assessment, medical schools will need to collaborate with residencies to determine competencies for graduating students. The objective of this study is to define the educational milestones for fourth-year medical students during an Internal Medicine sub-internship.

      Methods

      A cross-sectional Internet-based survey (with attention to validity evidence) was developed in early 2013 and administered to Internal Medicine attendings and Internal Medicine sub-interns working on an inpatient team at 3 academic medical centers. With the purpose to determine the milestones for sub-interns, items asked respondents what responsibilities a sub-intern could be entrusted to perform without direct supervision.

      Results

      Faculty responded that behaviors sub-interns could perform with indirect supervision were mostly at the “reporter” level, including completing a history and physical examination and collecting data such as test results. Other skills such as venipuncture and some communication skills such as calling consults, providing patient counseling, responding to pages, and creating discharge instructions were examples of tasks in which the majority of faculty felt that students were progressing toward unsupervised practice. Behaviors where the majority of faculty would always supervise a medical student performance included performance on the “interpreter” level, including interpreting electrocardiograms, significant physical examination findings, and laboratory results. Medical students less commonly noted needing supervision on the majority of the items when compared with faculty.

      Conclusion

      Tasks in the reporter domain such as taking a history, collecting medical records, and reporting results can be characterized as medical student milestones.

      Keywords

      Perspectives Viewpoints
      • Medical student sub-interns could perform with indirect supervision mostly at the “reporter” level, including completing a history and physical examination and collecting data such as test results.
      • Skills such as venipuncture and some communication skills such as calling consults, providing patient counseling, responding to pages, and creating discharge instructions were examples of tasks in which the majority of faculty felt that students were in need of some supervision.
      • Medical students responded that they needed less supervision on the majority of the items when compared with faculty.
      Independent licensed practitioners are the ultimate outcomes of the medical education system in the US.

      Joint Commission on Accreditation of Healthcare Organizations. About the Joint Commission. Available at: http://www.jointcommission.org/about_us/about_the_joint_commission_main.aspx. Accessed October 2, 2013.

      Thus, it is imperative that accreditation requirements ensure that schools and training programs graduate trainees who are able to practice medicine competently and independently. The determination of readiness and the degree of unsupervised care afforded by undergraduate medical education (UME) and graduate medical education (GME) programs continues to be an area of great deliberation.
      Over the past decade there has been a movement toward competency-based medical education, with the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project in 2002

      Accreditation Council for Graduate Medical Education. Outcome project. Available at: http://www.acgme.org/outcome/. Accessed October 2, 2012.

      and the CanMEDs Competency Framework from the Royal College of Physicians and Surgeons of Canada in 1996.

      Royal College of Physicians and Surgeons of Canada. The CanMEDS 2005 Physician Competency Framework. Available at: http://rcpsc.medical.org/canmeds/ CanMEDS2005/index.php. Accessed May 15, 2013.

      The intent is to enhance the medical profession's ability to meet the terms of the social contract and verify that graduates are competent to deliver safe and effective patient care.
      • Green M.L.
      • Aagaard E.M.
      • Caverzagie K.J.
      • et al.
      Charting the road to competence: developmental milestones for internal medicine residency training.
      Recently there has been increased awareness of the need to define and measure the intermediate steps (milestones) toward competency.
      • Swing S.R.
      • Beeson M.S.
      • Carraccio C.
      • et al.
      Educational milestone development in the first 7 specialties to enter the next accreditation system.
      • Sullivan G.
      • Simpson D.
      • Cooney T.
      • Beresin E.
      A milestone in the milestones movement: the JGME milestones supplement.
      This competency-based progression allows for the evaluation and feedback to learners at all levels (UME & GME).
      • Hauer K.E.
      • Kohlwes J.
      • Cornett P.
      • et al.
      Identifying entrustable professional activities in internal medicine training.
      Further, the progression can be assessed at various times, accompanied by increased autonomy and decreased supervision, otherwise known as entrustable professional activities.
      • Sterkenburg A.
      • Barach P.
      • Kalkman C.
      • et al.
      When do supervising physicians decide to entrust residents with unsupervised tasks?.
      • Ten Cate O.
      Competency-based education, entrustable professional activities, and the power of language.
      • Ten Cate O.
      Nuts and bolts of entrustable professional activities.
      • Wijnen-Meijer M.
      • van der Schaaf M.
      • Nillesen K.
      • et al.
      Essential facets of competence that enable trust in graduates: a delphi study among physician educators in the Netherlands.
      The move to competency-based medical education has been championed by the accreditors in graduate and continuing medical education (ACGME, American Board of Medical Specialties, and its specialty boards). Despite initial calls in 1999 by the Medical Student Outcomes Project,
      • The Medical School Objectives Writing Group
      Learning objectives for medical student education–guidelines for medical schools: report I of the Medical School Objectives Project.
      the Liaison Committee on Medical Education has not been prescriptive in setting objectives or competencies for UME. Meanwhile, some individual schools have made an initial attempt to articulate UME educational milestones.

      University of California–San Francisco. Interpersonal and communication skills. 2012. Available at: http://meded.ucsf.edu/sites/meded.ucsf.edu/files/documents/undergraduate-medical-education/md-competenciesinterpersonalcommskills.pdf. Accessed November 12, 2013.

      University of California–San Francisco. MD competencies. 2012. Available at: http://meded.ucsf.edu/ume/md-competencies. Accessed November 12, 2013.

      Most recently, many specialty-level milestones are being developed along the continuum of medical education from medical school through residency and into practice.

      Accreditation Council for Graduate Medical Education, American Board of Emergency Medicine. The Emergency Medicine Milestones Project. 2012. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/EmergencyMedicineMilestones.pdf. Accessed November 12, 2013.

      Accreditation Council for Graduate Medical Education, American Board of Pediatrics. The Pediatrics Milestone Project. 2013. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/PediatricsMilestones.pdf. Accessed November 12, 2013.

      Accreditation Council for Graduate Medical Education, American Board of Internal Medicine. The Internal Medicine Milestone Project. 2012. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/InternalMedicineMilestones.pdf. Accessed November 12, 2013.

      For UME education, the community must determine the milestones for the graduating medical student entering residency. Recently, the Association of American Medical Colleges has put forth a set of Core Entrustable Professional Activities for Entering Residency that may provide a road map for milestones.

      American Association of Medical Colleges. Core entrustable professional activities for entering residency. 2013. Available at: https://www.mededportal.org/icollaborative/resource/887. Accessed November 12, 2013.

      Sub-internships are key for students entering Internal Medicine residencies where there are elevated faculty and resident expectations and increased patient responsibilities placed on students.
      • Lyss-Lerman P.
      • Teherani A.
      • Aagaard E.
      • et al.
      What training is needed in the fourth year of medical school? Views of residency program directors.
      • Walling A.
      • Merando A.
      The fourth year of medical education: a literature review.
      This autonomy is less than is expected of residents, but how much supervision is required for students is often unclear. These experiences should serve as preparation for internship, where, during the first months of training it has been shown that the interns had very little supervision for many tasks.
      • Raymond M.R.
      • Mee J.
      • King A.
      • et al.
      What new residents do during their initial months of training.
      Thus, program directors expect that fourth-year students should be “at or near the same level of independence as interns.”
      • Lyss-Lerman P.
      • Teherani A.
      • Aagaard E.
      • et al.
      What training is needed in the fourth year of medical school? Views of residency program directors.
      • Langdale L.A.
      • Schaad D.
      • Wipf J.
      • et al.
      Preparing graduates for the first year of residency: are medical schools meeting the need?.
      The question becomes, which skills should students be expected to be allowed to perform with indirect supervision

      American Association of Medical Colleges. Core entrustable professional activities for entering residency. 2013. Available at: https://www.mededportal.org/icollaborative/resource/887. Accessed November 12, 2013.

      and some degree of autonomy? In the RIME (reporter–interpreter–manager-educator) schema, one might expect students to be able to be a reporter–and at times, an interpreter–reliably.
      • Rodriguez R.G.
      • Pangaro L.N.
      AM last page: mapping the ACGME competencies to the RIME framework.
      Further, the organization of Clerkship Directors in Internal Medicine has listed the performance criteria for sub-interns along the ACGME competencies, but has not yet mapped these criteria to milestones.

      Harrell HE. Primer to the internal medicine subinternship: a guide produced by the clerkship directors in internal medicine. Available at: http://www.im.org/Publications/PhysiciansInTraining/Documents/SubIPrimer.pdf. Accessed November 12, 2013.

      These documents highlight the importance of the fourth year to include the common clinical experiences that best prepare students to support the transition to graduate medical training and the assessment of competencies.
      While there is an expectation for increased responsibility and decreased supervision, what students can do with minimal supervision is not clear. It is perhaps from this uncertainty that there has been the need to add additional rotations that include capstone and boot camps.
      • Esterl Jr., R.M.
      • Henzi D.L.
      • Cohn S.M.
      Senior medical student “boot camp”: can result in increased self-confidence before starting surgery internships.
      Further, this variability in student preparedness has led to much consternation about the value and efficacy of the fourth year.
      • Langdale L.A.
      • Schaad D.
      • Wipf J.
      • et al.
      Preparing graduates for the first year of residency: are medical schools meeting the need?.
      Limited student experiences and variability are extenuated further by external pressures that have removed students from direct patient care, thus making it difficult to determine if sub-internship rotations still fulfill the purpose of ensuring readiness for internship and many aspects of medical practice to be done without direct supervision.
      • Nixon L.J.
      • Aiyer M.
      • Durning S.
      • et al.
      Educating clerkship students in the era of resident duty hour restrictions.
      Therefore, the goal of this study is to begin to define the educational milestones for fourth-year medical students during an internal medicine sub-internship. The objective is to define the tasks that students can perform without direct supervision, from the perspective of faculty and students. It is our expectation that the articulation of sub-internship milestones would be one of the first steps in defining the level 1 or medical student graduation milestones.

      Methods

      Survey Development

      In order to further understand what is expected from sub-interns in the specialty, we embarked on a process of measuring educational milestones for fourth-year medical students by focusing on what responsibilities a student can be entrusted to perform without direct supervision. Survey development and methodology is described below and started in January 2013. This study was reviewed by the University of Michigan's Institutional Review Board and was deemed exempt.
      For content validity, the survey was developed based on a thorough review of the milestone literature, including the resident-based Internal Medicine milestones.
      • Green M.L.
      • Aagaard E.M.
      • Caverzagie K.J.
      • et al.
      Charting the road to competence: developmental milestones for internal medicine residency training.
      • The Medical School Objectives Writing Group
      Learning objectives for medical student education–guidelines for medical schools: report I of the Medical School Objectives Project.

      Accreditation Council for Graduate Medical Education, American Board of Emergency Medicine. The Emergency Medicine Milestones Project. 2012. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/EmergencyMedicineMilestones.pdf. Accessed November 12, 2013.

      Accreditation Council for Graduate Medical Education, American Board of Pediatrics. The Pediatrics Milestone Project. 2013. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/PediatricsMilestones.pdf. Accessed November 12, 2013.

      Accreditation Council for Graduate Medical Education, American Board of Internal Medicine. The Internal Medicine Milestone Project. 2012. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/InternalMedicineMilestones.pdf. Accessed November 12, 2013.

      • Lyss-Lerman P.
      • Teherani A.
      • Aagaard E.
      • et al.
      What training is needed in the fourth year of medical school? Views of residency program directors.
      • Raymond M.R.
      • Mee J.
      • King A.
      • et al.
      What new residents do during their initial months of training.
      In addition, we queried 15 faculty and fourth-year students about items they considered activities performed autonomously for the most part for a sub-intern. Finally, the author group integrated this information and designed the survey (Appendix, available online). The survey was piloted and revised with special attention to responses and question stems. Through this process, content, response process, and internal structure validity evidence was attained. The survey content for both medical students and attending physicians was identical, but was modified to reflect their respective positions. There was also a section for qualitative comments.
      We surveyed Internal Medicine attending physicians who worked on an inpatient team with fourth-year medical students at 3 different university medical centers: Tulane University, Emory University, and the University of Michigan. Fourth-year medical students at the same institutions who had completed their sub-internships and matched into Internal Medicine were surveyed also.

      Survey Implementation and Analysis

      The survey was administered anonymously through Qualtrics (Provo, UT) with multiple e-mail reminders. The response rate was 46%. Descriptive statistics were reported. Tasks in which the majority of faculty would always supervise medical student work were not considered milestones, while those tasks with “> 50%,” “sometimes,” or “never” supervising might be considered milestones. Differences in responses between faculty and students were compared using the chi-squared statistic (SPSS 19; IBM, Armonk, NY). Faculty and student comments were analyzed qualitatively. Themes from the survey were reviewed by 2 of the authors independently. The final themes were reached through discussion until consensus was obtained.

      Results

      The results of the faculty and student surveys are found in Table 1. Faculty felt that items students could perform with indirect supervision were mostly at the “reporter” level, including completing a history and physical examination and collecting data such as test results. Other skills such as venipuncture and some communication skills such as calling consults, providing patient counseling, responding to pages, and creating discharge instructions were examples of tasks in which the majority of faculty felt that students were progressing toward unsupervised practice. These behaviors might be considered milestones.
      Table 1Responses of Faculty and Students
      ResponsibilityFaculty (%)Students (%)P-Value
      Using a Bonferroni correction to adjust for multiple comparisons, P < .002 would be significant.
      When caring for adult patients with an average sub-intern (fourth year medical student) on an inpatient Internal Medicine rotation, how often do you feel that the following activities performed by the student should be directly supervised (ie, being present or needing to repeat) by a more senior member of the care team (eg, house officer, attending physician).AlwaysSometimesNeverAlwaysSometimesNeverChi-squared of Faculty/Student
      Reporter level
       Complete patient history1975656530<.001
       Complete physical examination25723186914.047
       Collect medical records and report results7692733068<.001
       Collect current laboratory, imaging, and study information and report results155728103160.001
      Interpreter level (… in the context of the patient's problems or chief complaint)
       Recognize and interpret significant historical information4847521736.005
       Recognize and interpret significant physical examination findings5740327712.005
       Recognize and interpret significant laboratory results6134533616.006
       Recognize and interpret significant radiology results6632238593.006
       Recognize and interpret significant ECG result7426047522.006
       Develop and prioritize a comprehensive patient problem list5544232626.02
       Develop and prioritize a differential diagnosis with rationale for each patient problem5741235623.04
       Formulate basic plans based on differential diagnosis [manager]6233534633.004
      Skills
       Perform venipuncture27667424513N.s.
       Perform arterial blood gas6038269283N.s.
       Obtaining consent for advanced procedures70300523810.020
       Follow-up pending inpatient laboratory, imaging or study results and adjust care plans accordingly50482256411.005
      Communication
       Communicate patient care sign-out or handoff information with a covering team4552331618N.s.
       Receive admission sign-out or handoff information for a patient newly admitted by a covering team and acquired by the student5638736605.05
       Request initial consult with a specialist using a succinct patient history and appropriate clinical question18711256431.004
       Respond to nonemergent pages and discuss routine care plans with other members of the care team12741555541.004
       Provide counseling to patient, family, or caregiver regarding routine issues1282736631.001
       Discuss results of diagnostic tests and care plans with patient, family, or caregiver31690187111.01
       Conduct significant nonroutine discussions with patient, family, or caregiver6928346495.04
      Review hospital course and patient education/instructions with patient, family or caregiver at discharge2372587518.01
      Discuss hospital course and care plans with the patient's outpatient provider(s)17758107713N.s.
      Document an admission history and physical examination note63325195625<.001
      Document a daily progress note57368164738<.001
      Document a discharge summary note71263294922<.001
      Systems-based practice & patient safety
      Perform accurate medication reconciliation on admission and discharge4747740538N.s.
      Use a formulary or insurance information to control medication costs1775822727N.s.
      Identify a medical error, near-miss, or patient safety issue and report it to the appropriate party in a timely and accurate fashion6336267303N.s.
      Include important safety issues in the patient's routine care plan3363338583N.s.
      Enter patient care orders and ensure they are completed60392365311.009
      ECG = electrocardiogram; N.s. = not significant.
      Using a Bonferroni correction to adjust for multiple comparisons, P < .002 would be significant.
      Behaviors where the majority of faculty would always supervise a medical student performance included performance on the “interpreter” level, including interpreting electrocardiograms, significant physical examination findings, and laboratory results. Further, behaviors that are linked to systems issues such as documentation (history and physical examination, progress, and discharge notes) and orders were also behaviors in which the majority of faculty always supervise medical student performance.
      Faculty who always felt the need to supervise sub-interns noted that the following factors affected their need to supervise: billing or system constraints (21%), the need to provide patient care (60%), sub-intern cannot be entrusted reliably to complete the task (36%), and patient expectations (39%). Faculty could choose more than one factor. The practice site (ie, county setting or specific medical center) did not affect responses. Medical students less commonly noted they needed supervision on the majority of the items when compared with faculty (Table 1).
      There were 2 themes from the faculty comments. The first involved their expectations that it is the residents' responsibility to supervise students. “I expect the senior resident to be confirming key history, findings and results, but not necessarily directly supervising the entire process.” Second, faculty noted that how closely they would supervise students was based on how well they knew the students and how they judged their skills. “How comfortable I would be allowing the student to do this depends on the student.”
      There were 2 themes from the student comments. The first was recognition of the importance of supervision for patient safety. “I do think the senior house officer and attending should trust the sub-I but in any case where there is a more senior member of the team participating in patient care, it is my opinion that the most senior member should always repeat everything and make sure things are being done correctly.” The second theme was the need for supervision so as to provide feedback for improvement. “A supervised history and physical would be useful for feedback, but would only require around 2-3 episodes of observation.”

      Discussion

      In this study, we sought to define milestones for the graduating medical student. To accomplish this goal, we surveyed medical school faculty who supervise sub-interns on their internal medicine rotation as to what tasks they allowed students to perform with indirect supervision. Considering faculty responses, one can divide items into 2 major categories. The first category includes the behaviors in which the majority of the faculty would sometimes or never supervise medical student performance. These were mostly at the “reporter” level and included completing a history and physical examination and collecting data. These behaviors might be considered milestones. The second category included behaviors where the majority of faculty would always supervise a medical student performance. These behaviors include the performance on the “interpreter” level, including interpreting electrocardiograms, significant physical examination findings, and laboratory results as well as systems issues such as documentation and orders. These behaviors are likely above the competency level of a sub-intern.
      The initial internal medicine milestone group adopted the Dreyfus model of skill acquisition as a framework for developing milestones.
      • Green M.L.
      • Aagaard E.M.
      • Caverzagie K.J.
      • et al.
      Charting the road to competence: developmental milestones for internal medicine residency training.

      Accreditation Council for Graduate Medical Education, American Board of Internal Medicine. The Internal Medicine Milestone Project. 2012. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/InternalMedicineMilestones.pdf. Accessed November 12, 2013.

      The competent learner considers both context and situational elements, but is also able to organize and prioritize information on which to base a decision.
      • Green M.L.
      • Aagaard E.M.
      • Caverzagie K.J.
      • et al.
      Charting the road to competence: developmental milestones for internal medicine residency training.
      • Dreyfus S.E.
      University of California BORC
      Formal Models Vs. Human Situational Understanding: Inherent Limitations on the Modeling of Business Expertise.
      • Dreyfus H.L.
      • Dreyfus S.E.
      Putting computers in their proper place: analysis versus intuition in the classroom.
      Based on these delineations of skill acquisition, one would expect a medical student as s/he becomes an intern to have achieved competency in some domains. Nonetheless, the medical system requires a certain degree of attending supervision of medical student performance for the purposes of patient safety, student education, and reimbursement.
      Competency without the act of entrustment makes decisions of independent practice difficult.
      • Ten Cate O.
      Competency-based education, entrustable professional activities, and the power of language.
      • Ten Cate O.
      Nuts and bolts of entrustable professional activities.
      For example, discussing a patient's hospital course and care plans with the patient's outpatient provider requires the skills of many of the ACGME competencies, including patient care, communication, and systems-based practice. The entrustment of activity by a student would describe features of work, whereas competencies are features of the trainee. In addition, the findings of this study of sub-intern milestones complements the list developed by the Association of American Medical Colleges Core Entrustable Professional Activities for Entering Residency (CEPAER) group, as shown in Table 2. However, there were some differences; for example, communication skills such as counseling of patients and use of a formulary appear to be a higher level than the CEPAER Entrustable Professional Activities. This is not surprising, as the CEPAER Entrustable Professional Activities are for all graduating residents, while these milestones are for Internal Medicine sub-interns who might be held to a higher level. Nonethless, the overlap in milestones and Entrustable Professional Activities lends credence to the idea that there are likely a set of responsibilities that should be expected of graduates.

      American Association of Medical Colleges. Core entrustable professional activities for entering residency. 2013. Available at: https://www.mededportal.org/icollaborative/resource/887. Accessed November 12, 2013.

      The clarification of the students' responsibilities will need to continue to be elucidated.
      Table 2Internal Medical Student Milestones Mapper to Core Entrustable Professional Activities for Entering Residency
      MilestoneEntrustable Professional Activity
      Complete patient historyEntrustable Professional Activity 1: Gather a history and perform a physical examination
      Complete physical examination
      Collect medical records and report results
      Collect current laboratory, imaging, and study information and report results
      Recognize and interpret significant historical information
      Recognize and interpret significant physical examination findings
      Recognize and interpret significant laboratory resultsEntrustable Professional Activity 3: Recommend and interpret common diagnostic and screening tests
      Recognize and interpret significant radiology results
      Recognize and interpret significant ECG result
      Formulate basic plans based on differential diagnosis [manager]
      Develop and prioritize a comprehensive patient problem listEntrustable Professional Activity 2: Develop a prioritized differential diagnosis and select a working diagnosis following a patient encounter
      Develop and prioritize a differential diagnosis with rationale for each patient problem
      Perform venipunctureEntrustable Professional Activity 12: Perform general procedures of a physician
      Perform arterial blood gasNot covered in Entrustable Professional Activity 12, but might be considered in a specialty specific procedure expectation
      Obtaining consent for advanced proceduresEntrustable Professional Activity 11: Obtain informed consent for tests or procedures that the day 1 intern is expected to perform or order without supervision
      Follow-up pending inpatient laboratory, imaging, or study results and adjust care plans accordinglyEntrustable Professional Activity 3: Recommend and interpret common diagnostic and screening tests
      Communicate patient care sign-out or handoff information with a covering teamEntrustable Professional Activity 8: Give or receive a patient handover to transition care responsibility to another health care provider or team
      Receive admission sign-out or handoff information for a patient newly admitted by a covering team and acquired by the studentEntrustable Professional Activity 8: Give or receive a patient handover to transition care responsibility to another health care provider or team
      Request initial consult with a specialist using a succinct patient history and appropriate clinical questionEntrustable Professional Activity 9: Participate as a contributing and integrated member of an interprofessional team (includes communicating with consultants)
      Respond to nonemergent pages and discuss routine care plans with other members of the care teamEntrustable Professional Activity 9: Participate as a contributing and integrated member of an interprofessional team
      Provide counseling to patient, family, or caregiver regarding routine issuesEntrustable Professional Activity 3: Recommend and interpret common diagnostic and screening tests (and Entrustable Professional Activity 4, 11, 12)
      Discuss results of diagnostic tests and care plans with patient, family, or caregiver
      Conduct significant nonroutine discussions with patient, family, or caregiver
      Review hospital course and patient education/instructions with patient, family or caregiver at discharge
      Discuss hospital course and care plans with the patient's outpatient provider (s)Entrustable Professional Activity 9: Participate as a contributing and integrated member of an interprofessional team
      Document an admission history and physical examination noteEntrustable Professional Activity 5: Provide documentation of a clinical encounter in written or electronic format
      Document a daily progress note
      Document a discharge summary note
      Perform accurate medication reconciliation on admission and discharge
      Use a formulary or insurance information to control medication costsEntrustable Professional Activity 3 Bulleted List: Recommend and interpret common diagnostic and screening tests (addresses costs of tests but not of medications) & Entrustable Professional Activity 4: Enter and discuss patient orders/prescriptions (does not address costs)
      Identify a medical error, near-miss, or patient safety issue and report it to the appropriate party in a timely and accurate fashionEntrustable Professional Activity 13: Identify system failures and contribute to a culture of safety and improvement
      Include important safety issues in the patient's routine care plan
      Enter patient care orders and ensure they are completedEntrustable Professional Activity 4: Enter and discuss patient orders/prescriptions
      The medical student milestones align with the “starting point” of resident milestones in terms of domains. There are several important differences. For the most part, content was similar, but specificity and wording differed between the Internal Medicine (IM) medical student milestones compared with the residents. First, a couple of the medical student milestones were not explicitly covered in the IM resident ones. For example, the IM milestones for residents include skill in performing common procedures, although they do not clarify the types of procedures, such as venipuncture and arterial blood gas. Likewise, medicine reconciliation is not explicit. Second, the IM resident milestones contain a number of domains not included in the medical student ones; these include areas such as a number of the practice-based performance and improvement and professionalism competencies. These might be included for medical students, but may also be deferred for a later time when there is more of a longitudinal relationship that would lead to more accurate assessment. Finally, our survey responses clearly indicate that sub-interns likely have skills that do not require constant supervision, but the ACGME's suggestion is that across all milestones, the beginning intern “requires direct supervision to ensure patient safety and quality care.”

      Accreditation Council for Graduate Medical Education, American Board of Internal Medicine. The Internal Medicine Milestone Project. 2012. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/InternalMedicineMilestones.pdf. Accessed November 12, 2013.

      This demonstrates the tension between direct supervision through direct observation and the current practice of entrusting students with indirect supervision for many responsibilities.
      It is notable from our results that there were significant discrepancies between faculty and students about the level of entrustment, with students less frequently responding that they always require supervision. This pattern has been seen with residents in IM and Anesthesia, where the trainees thought they should be more independent than did faculty.
      • Hauer K.E.
      • Kohlwes J.
      • Cornett P.
      • et al.
      Identifying entrustable professional activities in internal medicine training.
      • Sterkenburg A.
      • Barach P.
      • Kalkman C.
      • et al.
      When do supervising physicians decide to entrust residents with unsupervised tasks?.
      Faculty more commonly chose “sometimes,” meaning that there is variation when entrustment is appropriate. This came out further as faculty noted that the decision to entrust is based on knowing the student. This might be described as “trust but verify,” which suggests that it is appropriate to trust trainees, but the faculty will verify whether the trust is warranted by repeating key portions of medical care. This study did not determine the number of competency-based observations needed on the part of faculty to ensure that a student no longer needs direct observation.
      Further, while faculty in our study felt the need to supervise students, there are a number of studies documenting the lack of direct observation of medical students in various clerkships.
      • Fromme H.B.
      • Karani R.
      • Downing S.M.
      Direct observation in medical education: a review of the literature and evidence for validity.
      • Mazor K.M.
      • Holtman M.C.
      • Shchukin Y.
      • et al.
      The relationship between direct observation, knowledge, and feedback: results of a national survey.
      • Pulito A.R.
      • Donnelly M.B.
      • Plymale M.
      • Mentzer J.R.M.
      What do faculty observe of medical students' clinical performance?.
      We know from the graduation questionnaire that students frequently may not be observed. In addition, faculty will need to not only observe students to ensure their skill, but also be trained in competency-based assessment rather than normative grading for medical students. The question that arises is: How many observations are needed to ensure competence so that the faculty can trust the student without verification by direct supervision? Resident supervision may help to contribute to the assessment of competences. Mischler describes sub-internship competencies with a summative observed standardized clinical examination that might provide a reference for these decisions, but more research is needed.
      • Mischler M.
      • Miller G.
      • Aldag J.
      • Aiyer M.K.
      Last chance to observe: assessing residency preparedness following the 4th-year subinternship.
      The study has several limitations. We sampled 3 settings, but these may not be representative of other medical schools and institutions. Further, there may be response bias. The questions are clearly based on opinions of the faculty and students. From the qualitative comments, it may be that if the faculty knew more about the student or had worked with the student previously, they would be more willing to entrust him or her with more significant activities without direct supervision; perhaps the hypothetical “average” sub-intern referred to in the survey introduction did not provide a suitable anchor.
      In conclusion, there are some tasks in the reporter domain such as taking a history, collecting medical records, and reporting results that can be characterized as medical student milestones. However, most interpreter and many advanced communication and systems-based practice skills are considered too advanced for a medical student milestone. More work is needed to better understand and further develop the medical student milestones.

      Appendix

      Digital Resources: Survey Introduction (Faculty Version)

      The American Board of Internal Medicine (ABIM) and Accreditation Council for Graduate Medical Education (ACGME) have initiated the development of milestones for internal medicine residency training as means to promote competency-based resident education in internal medicine. Milestones are learning objectives that describe specific skills and knowledge, which lead to the development of competence.
      Milestones for medical students have not yet been instituted. Our goal is to determine which activities should be considered milestones for a fourth year medical student on an internal medicine sub-internship.
      This is an anonymous survey and has been IRB approved.

      Section One

      When caring for adult patients with an average sub-intern (fourth year medical student) on an inpatient Internal Medicine rotation, how often do you feel that the following activities performed by the student should be directly supervised (ie, being present or needing to repeat) by a more senior member of the care team (eg, house officer, attending physician)?
      Tabled 1
      AlwaysSometimesNever
      Perform a complete patient history.
      Perform a complete physical examination.
      Collect medical records (historical laboratory/imaging/study results, outside hospital documentation) and report results.
      Collect current laboratory, imaging, and study information and report results.
      Recognize and interpret significant historical information in the context of the patient's problems or chief complaint (eg nausea, vomiting, diarrhea, and continued use of diuretics likely contribute to the patient's acute kidney injury).
      Recognize and interpret significant physical examination findings in the context of the patient's problems or chief complaint (eg elevated jugular venous distention, rales, and lower extremity edema are consistent with heart failure).
      Recognize and interpret significant laboratory results in the context of the patient's problems or chief complaint (eg decrease in hemoglobin by 3 grams is significant in the setting of hematochezia).
      Recognize and interpret significant radiology results in the context of the patient's problems or chief complaint (eg chest radiograph reveals a lobar infiltrate in a patient with fever, dyspnea, and cough).
      Recognize and interpret significant EKG results in the context of the patient's problems or chief complaint (eg electrocardiogram reveals ST segment elevation in a patient with chest pain; telemetry reveals atrial fibrillation with rapid ventricular response in a patient with palpitations).
      Develop and prioritize a comprehensive patient problem list.
      Develop and prioritize a differential diagnosis with rationale for each patient problem.
      Formulate basic plans based on differential diagnosis (eg in a patient with new dyspnea and hypoxia, monitor vital signs and fluid balance, order complete blood count, cardiac biomarkers, arterial blood gas, electrocardiogram, chest radiograph, and provide supplemental oxygen).
      Perform venipuncture.
      Perform arterial blood gas.
      Obtaining consent for advanced procedures (eg lumbar puncture, thoracentesis).
      Follow-up pending inpatient laboratory, imaging, or study results and adjust care plans accordingly.
      Communicate patient care signout or handoff information with a covering team (includes providing signout to and receiving signout from a covering team).
      Receive admission signout or handoff information for a patient newly admitted by a covering team and acquired by the student.
      Request initial consult with a specialist using a succinct patient history and appropriate clinical question.
      Respond to non-emergent pages and discuss routine care plans with other members of the care team (eg consultant follow-up, nursing staff, ancillary staff).
      Provide counseling to patient, family, and/or caregiver regarding routine issues (eg smoking cessation, diet modification).
      Discuss results of diagnostic tests and care plans with patient, family, and/or caregiver.
      Conduct significant non-routine discussions with patient, family, and/or caregiver (eg discussions regarding code status, clinical deterioration, end-of-life care, etc.).
      Review hospital course and patient education/instructions with patient, family, or caregiver at discharge.
      Perform accurate medication reconciliation on admission and discharge.
      Discuss hospital course and care plans with the patient's outpatient provider(s).
      Use a formulary or insurance information to control medication costs.
      Identify a medical error, near-miss, or patient safety issue and report it to the appropriate party in a timely and accurate fashion.
      Include important safety issues in the patient's routine care plan (eg discontinue Foley catheter if appropriate to prevent catheter-associated urinary tract infection, recognize a patient at risk for falls, prevent sacral ulceration with regular position changes).
      Enter patient care orders and ensure they are completed.
      Document an admission history and physical examination note.
      Document a daily progress note.
      Document a discharge summary note.
      Each section was followed by the question: If you always feel the need to supervise one of the activities above, why? This was followed by a comment box.
      • Billing or system constraints
      • Assist with patient care
      • Sub-I cannot be entrusted reliably complete the task
      • Patient expectations
      • Other

      Demographic Questions

      What institution are you primarily affiliated with?
      What type of hospital do you primarily attend general medicine wards with medical students?
      What type of hospital do you primarily attend general medicine wards with medical students?
      What type of hospital do you primarily attend general medicine wards with medical students?

      References

      1. Joint Commission on Accreditation of Healthcare Organizations. About the Joint Commission. Available at: http://www.jointcommission.org/about_us/about_the_joint_commission_main.aspx. Accessed October 2, 2013.

      2. Accreditation Council for Graduate Medical Education. Outcome project. Available at: http://www.acgme.org/outcome/. Accessed October 2, 2012.

      3. Royal College of Physicians and Surgeons of Canada. The CanMEDS 2005 Physician Competency Framework. Available at: http://rcpsc.medical.org/canmeds/ CanMEDS2005/index.php. Accessed May 15, 2013.

        • Green M.L.
        • Aagaard E.M.
        • Caverzagie K.J.
        • et al.
        Charting the road to competence: developmental milestones for internal medicine residency training.
        J Grad Med Educ. 2009; 1: 5-20
        • Swing S.R.
        • Beeson M.S.
        • Carraccio C.
        • et al.
        Educational milestone development in the first 7 specialties to enter the next accreditation system.
        J Grad Med Educ. 2013; 5: 98-106
        • Sullivan G.
        • Simpson D.
        • Cooney T.
        • Beresin E.
        A milestone in the milestones movement: the JGME milestones supplement.
        J Grad Med Educ. 2013; 5: 1-4
        • Hauer K.E.
        • Kohlwes J.
        • Cornett P.
        • et al.
        Identifying entrustable professional activities in internal medicine training.
        J Grad Med Educ. 2013; 5: 54-59
        • Sterkenburg A.
        • Barach P.
        • Kalkman C.
        • et al.
        When do supervising physicians decide to entrust residents with unsupervised tasks?.
        Acad Med. 2010; 85: 1408-1417
        • Ten Cate O.
        Competency-based education, entrustable professional activities, and the power of language.
        J Grad Med Educ. 2013; 5: 6-7
        • Ten Cate O.
        Nuts and bolts of entrustable professional activities.
        J Grad Med Educ. 2013; 5: 157-158
        • Wijnen-Meijer M.
        • van der Schaaf M.
        • Nillesen K.
        • et al.
        Essential facets of competence that enable trust in graduates: a delphi study among physician educators in the Netherlands.
        J Grad Med Educ. 2013; 5: 46-53
        • The Medical School Objectives Writing Group
        Learning objectives for medical student education–guidelines for medical schools: report I of the Medical School Objectives Project.
        Acad Med. 1999; 74: 13-18
      4. University of California–San Francisco. Interpersonal and communication skills. 2012. Available at: http://meded.ucsf.edu/sites/meded.ucsf.edu/files/documents/undergraduate-medical-education/md-competenciesinterpersonalcommskills.pdf. Accessed November 12, 2013.

      5. University of California–San Francisco. MD competencies. 2012. Available at: http://meded.ucsf.edu/ume/md-competencies. Accessed November 12, 2013.

      6. Accreditation Council for Graduate Medical Education, American Board of Emergency Medicine. The Emergency Medicine Milestones Project. 2012. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/EmergencyMedicineMilestones.pdf. Accessed November 12, 2013.

      7. Accreditation Council for Graduate Medical Education, American Board of Pediatrics. The Pediatrics Milestone Project. 2013. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/PediatricsMilestones.pdf. Accessed November 12, 2013.

      8. Accreditation Council for Graduate Medical Education, American Board of Internal Medicine. The Internal Medicine Milestone Project. 2012. Available at: http://www.acgme.org/acgmeweb/Portals/0/PDFs/Milestones/InternalMedicineMilestones.pdf. Accessed November 12, 2013.

      9. American Association of Medical Colleges. Core entrustable professional activities for entering residency. 2013. Available at: https://www.mededportal.org/icollaborative/resource/887. Accessed November 12, 2013.

        • Lyss-Lerman P.
        • Teherani A.
        • Aagaard E.
        • et al.
        What training is needed in the fourth year of medical school? Views of residency program directors.
        Acad Med. 2009; 84: 823-829
        • Walling A.
        • Merando A.
        The fourth year of medical education: a literature review.
        Acad Med. 2010; 85: 1698-1704
        • Raymond M.R.
        • Mee J.
        • King A.
        • et al.
        What new residents do during their initial months of training.
        Acad Med. 2011; 86: S2059-S2062
        • Langdale L.A.
        • Schaad D.
        • Wipf J.
        • et al.
        Preparing graduates for the first year of residency: are medical schools meeting the need?.
        Acad Med. 2003; 78: 39-44
        • Rodriguez R.G.
        • Pangaro L.N.
        AM last page: mapping the ACGME competencies to the RIME framework.
        Acad Med. 2012; 87: 1781
      10. Harrell HE. Primer to the internal medicine subinternship: a guide produced by the clerkship directors in internal medicine. Available at: http://www.im.org/Publications/PhysiciansInTraining/Documents/SubIPrimer.pdf. Accessed November 12, 2013.

        • Esterl Jr., R.M.
        • Henzi D.L.
        • Cohn S.M.
        Senior medical student “boot camp”: can result in increased self-confidence before starting surgery internships.
        Curr Surg. 2006; 63: 264-268
        • Nixon L.J.
        • Aiyer M.
        • Durning S.
        • et al.
        Educating clerkship students in the era of resident duty hour restrictions.
        Am J Med. 2011; 124: 671-676
        • Dreyfus S.E.
        • University of California BORC
        Formal Models Vs. Human Situational Understanding: Inherent Limitations on the Modeling of Business Expertise.
        Operations Research Center, University of California, Berkeley, CA1981
        • Dreyfus H.L.
        • Dreyfus S.E.
        Putting computers in their proper place: analysis versus intuition in the classroom.
        in: Sloan D. The Computer in Education: A Critical Perspective. Teachers' College Press, Columbia, NY1984: p. 40-63
        • Fromme H.B.
        • Karani R.
        • Downing S.M.
        Direct observation in medical education: a review of the literature and evidence for validity.
        Mt Sinai J Med. 2009; 76: 365-371
        • Mazor K.M.
        • Holtman M.C.
        • Shchukin Y.
        • et al.
        The relationship between direct observation, knowledge, and feedback: results of a national survey.
        Acad Med. 2011; 86: S2063-S2068
        • Pulito A.R.
        • Donnelly M.B.
        • Plymale M.
        • Mentzer J.R.M.
        What do faculty observe of medical students' clinical performance?.
        Teach Learn Med. 2006; 18: 99-104
        • Mischler M.
        • Miller G.
        • Aldag J.
        • Aiyer M.K.
        Last chance to observe: assessing residency preparedness following the 4th-year subinternship.
        Teach Learn Med. 2013; 25: 242-248

      Linked Article

      • Successful Sub-Internships Promote Internal Medicine Career Choice
        The American Journal of MedicineVol. 130Issue 6
        • Preview
          We read with interest the recently published article by Santen et al.1 Tasks on the reporter level like history taking and reporting results were considered milestones in internal medicine (IM) sub-internships. Medical students expected decreased supervision and increased responsibility, compared with faculty. Moreover, sub-internships may influence later career choice, but there is still a strong need for evaluation data.2,3
        • Full-Text
        • PDF