The American Journal of Medicine
Volume 123, Issue 3 , Pages 286-290, March 2010

Measuring Resident Hours by Tracking Interactions with the Computerized Record

  • Daniel Shine, MD

      Affiliations

    • Department of Medicine, New York University Langone Medical Center, New York, NY
    • Corresponding Author InformationReprint requests should be addressed to Daniel Shine, MD, TCH 1626, New York University Langone Medical Center, 550 First Avenue, New York, NY 10016
    • ,
  • Ellen Pearlman, MD

      Affiliations

    • Department of Medicine, New York University Langone Medical Center, New York, NY
    • ,
  • Brendan Watkins, BA

      Affiliations

    • Medical Center Information Technology, New York University Langone Medical Center, New York, NY

Article Outline

 

The impact of resident duty hours on clinical and educational outcomes continues to concern professional and government oversight groups. Duty hours regulations have both stimulated and responded to a growing literature; but even after considerable study, the relationships remain uncertain among sleep, fatigue, effective education, hospital working conditions, hand-offs, patient safety, and resident burn-out. To be persuasive, reports investigating these relationships must accurately and completely measure the independent variable: resident duty hours. The almost universal practice of deriving duty hours from retrospectively completed time-cards, whether actual or computerized, has been criticized as potentially biased and poorly reproducible.

Perspectives Viewpoints

 


•Regulatory oversight of resident work schedules has obliged training programs to monitor resident hours closely. Work time is usually measured by self-report using resident time-cards, actual or computerized.

•This study compares measurement of hours using paper, resident-completed time-cards to automated reports of resident interactions with the institution's electronic medical record (EMR).

•EMR surveillance yielded information similar to time-cards with greater ease and timeliness.

Regulation of resident duty hours began as a legislative initiative in New York 25 years ago.1 This local issue appeared nationally in 2002 when the Occupational Safety and Health Administration, under petition by 3 concerned groups, first considered limiting hours for the sake of both resident and patient safety.2, 3, 4 Anticipating federal regulation, the Accreditation Council for Graduate Medical Education, a professional certification body, issued requirements the following year and subsequently to all training programs, regardless of specialty.5, 6, 7, 8 Most recently, the Institute of Medicine, an influential watchdog organization, has recommended that resident hours be further reduced to approach requirements in Europe. The European Working Time Directive requires that medical training be limited to 48 hours per week with 11 hours of rest in each 24-hour period.7, 8

Lack of sleep impairs vigilance and task-oriented performance.9, 10, 11 Less clear is the effect of fatigue on cognitive and team-based activities, such as rounding. Other workplace variables may bear as much on impairment and fatigue as do hours of sleep.12, 13

At least a dozen investigators have systematically examined changes in patient outcomes before and after Accreditation Council for Graduate Medical Education duty hours regulation implementation.14, 15, 16, 17, 18 Review of 7 such studies in 2004 found insufficient data to demonstrate effectiveness.14 Since then, one large study in pediatric patients detected significantly less resident “burn-out,” but found no effect of the new regulations on hours of sleep or total hours of work. The authors reported a small increase in medication errors after the new hours.15 On the other hand, 3 recent studies demonstrated medical (but not surgical) mortality improvement among Medicare beneficiaries and in teaching hospitals (but not in non-teaching hospitals) comparing the years before and after reduction of resident hours.16, 17, 18

Such disparity among studies examining the outcomes of regulation suggests that the extent (and perhaps therefore the effect) of decreasing resident hours may be local and variable. Only a credible method for measuring hours will confirm or refute this suggestion or provide a direct link between hours and outcomes. Several authors have suspected, and one has measured, substantial unreliability of time-card data.19

Back to Article Outline

Materials and Methods 

We aimed to validate self-report of duty hours (and of their distribution within the week) during a busy rotation in the medical intensive care unit using a more objective method: surveillance of all resident interactions with the electronic medical record.

The Internal Medicine Residency Training Program at New York University Langone Medical Center rotates 164 residents through thirteen 28-day blocks per year at the University Hospital and two affiliated institutions. Typically, 2 blocks are spent in the medical intensive care unit. During 5 successive blocks between December 2008 and March 2009 (as previously for several years), we distributed index cards to all residents every 2 weeks during geriatrics, general medicine, intensive care, cardiology, hematology/oncology, and intensive care rotations at the University Hospital. In accordance with previous practice, residents were asked to be both truthful and precise in recalling the numbers of hours worked each day during the previous 2 weeks. Cards were distributed and collected on the same day.

Residents were unaware that data also were collected from the electronic medical record system (Sunrise Clinical Manager, Eclypsis Corporation, Atlanta, Ga), capturing the times of all their interactions with the system. Interactions included entering orders, viewing results, writing notes, and opening notes to read them. We hypothesized that the data-rich and quickly changing clinical environment in our medical intensive care unit demanded frequent interaction with the electronic medical record; we proposed that residents could not therefore be engaged in patient care in this milieu without regular recourse to the computer.

Residents were considered working if no 6-hour period passed without their contacting the institution's electronic medical record. We recorded a resident as having left work at the time of last computer interaction before a 6-hour hiatus. An Excel (Microsoft Corp, Redmond, WA) spreadsheet with preprogrammed calculated columns permitted rapid determination of the beginning and end of resident work days by the noted criteria. In addition, the spreadsheet automatically summed the number of violations specific to each of the 4 hour-related rules set by the Residency Review Committee in Internal Medicine following Accreditation Council for Graduate Medical Education standards.20

The spreadsheet, written by one of our nontechnical authors (D.S.), required approximately 6 hours to construct and test, as well as 15 minutes to apply to each monthly data set. The downloaded electronic medical record data were pasted into the spreadsheet and required 10 minutes for our technical author (B.W.) to enter resident names and run each monthly data set.

At the end of the 4-month study period, we compared total resident hours and occurrence of violations between electronic medical record surveillance and resident self-report. Fractional hours on time-cards were rounded to whole numbers. Violations were expressed as a fraction in which the numerator was the measured number of violations and the denominator was the number of opportunities for violation, based on the number of days for which each resident submitted time-cards. Table 1 defines the 4 violation types and the method for applying those definitions to time-cards and to surveillance downloads from the electronic medical record.

Table 1. Explanation and Details of Hours Violations
Brief Regulation NameSummary of RegulationHow Regulation Was Applied to EMRHow Regulation Was Applied to Time-CardsViolations among Residents Submitting Cards (n=28)Violations Among Residents Not Submitting Cards (n=8)
a Potential Violationsb Observed Violations by Time-Card (%)c Observed Violations by EMR (% of Potential)b/a vs c/a P Value by Chi-square with Continuity Correction (Fisher Exact Test*)d Potential Violations on Dates with Computer Contacte Observed Violations by EMR (% of Potential)e/d vs c/a P Value by Chi-square with Continuity Correction (Fisher Exact Test*)
80-hResidents must not spend>80 h per week in a training site. Weekly hours may be averaged over the individual rotation.Starting with the first scheduled day in MICU on which there was computer activity, working periods were continuously summed until exactly 1 week later, when the process began again. Any week with summed working periods>80 h was considered in violation.Hours were tallied for each week, starting with days 1 and 8 on the card. Any week>80 h was considered in violation.715(7%)1(1.4%)P=.21*140(0%)P=.4*
27-hResidents must not spend>24 h on call. A 3-h period is allowed for signout of patients.Any continuous period>27 hours was considered in violation.Any continuous period>27 h was considered in violation.1549(5.8%)9(5.8%)P=1.0322(6.3%)P=.6*
24-hResidents must have 24 h away from all training sites in each week.Any week (as defined for 80-h violations above) in which there was not a single≥24-h period with no computer activity was considered in violation.Any week (days 1-7 or 8-14) on the time-card in which there was not a stated or easily calculable 24-h period off was considered in violation.710(0%)2(2.8%)P=.5*140(0%)P=.7*
10-hResidents must have 10 h away from all training sites after each shift.Time of first EMR interaction after a 6-h hiatus is considered a violation if it is > 10 h from the time of the last interaction.Time intervals > 10 h between reported end of a shift and reported start of another is considered a violation.4621(0.2%)3(0.6%)P=.4*9710(3.1%)P=.06*

EMR=electronic medical record; MICU=medical intensive care unit.

Definitions of hours regulations, electronic medical record, and time-card criteria for violation used in the study, and distribution of violations and percentage occurrence using electronic medical record surveillance and time-card review.

For residents submitting time-cards, we calculated potential violations and reported data only for those days on which the cards were submitted. We tested the significance of differences in occurrence of individual violation types between the 2 methods using the Fisher exact test (for<6 occurrences) and otherwise chi-square testing with continuity correction. For residents not submitting any cards, we calculated potential violations and reported hours data for days that were scheduled and on which electronic medical record reflected any interaction. No data were reported for days left undocumented by residents submitting incomplete time-cards.

Thirty-six residents rotated through the medical intensive care unit during the 4 study months, accounting for 744 scheduled resident days. Twenty-eight residents returned time-cards, accounting for 462 days and ranging from 11% to 100% of scheduled days. Eighteen responding residents reported all their scheduled days, and 4 residents reported less than one half of their scheduled days. Eight residents did not respond at all. Relying only on electronic medical record data, we tested for significance differences in occurrence of each violation type between those residents who did and did not hand in any cards.

Back to Article Outline

Results 

Responding residents reported a total 4383 hours; the electronic medical record surveillance report returned 4062 for these same days, a 7.3% discrepancy. Considering only scheduled days for which time-cards had been submitted, there were 71 opportunities to violate the 80-hour work week rule and the same number of opportunities to violate the standard of 1 day off weekly. There were 154 opportunities to violate the 27-hour rule and 462 opportunities to violate the requirement for 10 hours off between shifts (Table 1). The 758 possible total violations (1.9% of which actually occurred according to time-cards) yielded 80% power to detect a significant (P < .05) difference from the time-cards method it frequency of violation by electronic surveillance was less than 0.5% or more than 4.5%.

On review of time-cards, there were five 80-hour violations (7%), nine 27-hour violations (5.8%), no 24-hour violations (0%), and one 10-hour violation (0.2%). There were therefore a total of 15 time-card violations (1.9%). By electronic medical record surveillance there was one 80-hour violation (1.4%), nine 27-hour violations (5.8%), two 24-hour violations (2.8%), and three 10-hour violations (0.6%). Total violations by electronic medical record were 15 (1.9%). Occurrence of individual violations did not differ significantly between methods.

Among the 8 residents who did not return time-cards, there were no 80-hour violations (0%), two 27-hour violations (6.3%), no 24-hour violations (0%), and three 10-hour violations (3.1%). Total violations were therefore 5 (5.2%). There was no significant difference in occurrence of any violation type between residents who did and did not return any cards, although occurrence of 10-hour violations was 3% among individuals who did not return cards and 0.6% among residents who did return cards (P=.06).

Electronic medical record surveillance and time-card completion were approximately equivalent methods for measuring resident duty hours and occurrence of hours violations. Average total hours were 7.3% higher by time-cards, and total violations were identical. Electronic medical record surveillance had the advantages of ready availability and more complete data. The 8 noncompliant residents cannot be assumed to resemble the compliant 28 residents. It is easy to imagine, for example, that compliance in returning cards might have been lower in residents with more frequent hours violations. In fact, we found a trend toward more 10-hour violations among noncompliant residents.

A 7.3% disparity was not unexpected. The Accreditation Council for Graduate Medical Education regulations require not only that residents go off duty at the correct time but also that they physically leave the training site. We reasoned that a resident using the computer is always at work (high positive predictive value), but that a resident not using the computer may be still in the hospital (lower negative predictive value). The electronic medical record surveillance method is thus likely only to underestimate duty hours. Detected violations by electronic medical record provide strong evidence that a problem exists; apparent compliance measured by electronic surveillance, on the other hand, cannot entirely verify compliance.

Back to Article Outline

Limitations 

Limitations of this study include its performance at a single institution with an advanced electronic medical record and among only those residents who were assigned to the medical intensive care unit of the University Hospital. It is likely that residents rotating in areas where the patients are less ill will interact less frequently with the electronic medical record; underestimates of hours might therefore be greater than in a medical intensive care unit. The extent of additional underestimation will determine how much less useful this method is on a ward rotation than it seemed to be in an intensive care setting.

Electronic medical record surveillance is easily employed in hospitals with developed systems. This method is inexpensive, provides information about residents who do not return cards, produces real-time results, and requires little special expertise. However, it is but one of several possible solutions to the alleged uncertainties of time-cards. Electronic monitoring of transmitter-fitted resident badges might yield times of coming and going, and the addition of global positioning technology might even monitor location within the hospital. Expense may be a limiting factor. Malfunction and failure to wear the badge recreates the time-cards problem of interpreting missing data. Signing in and out is a far cheaper alternative, but subject to noncompliance.

Back to Article Outline

Conclusions 

Our finding that residents—compared with an independent and objective data source—accurately recorded their own hours and violations must be confirmed in other institutions and especially in less intensive clinical settings. This study suggests a possible alternative to time-cards but also provides validation for their use and therefore supports a method frequently used in published studies to measure the impact of resident duty hours on patient and resident outcomes.

Back to Article Outline

Acknowledgments 

The authors thank John Jackson Braider for assistance in editing the article.

Back to Article Outline

References 

  1. NY Comp. Codes R. & Regs. Section 405.4.2002.
  2. Public Citizen Health Research Group. Medical residents; work hours. New York: Public Citizen; 2003;www.citizen.org/hrg/healthcare/articles.cfm?ID=6666Accessed July 2, 2009
  3. American Medical Student Association. The Resident Work Hour Issue. www.amsa.org/rwh/index.cfmAccessed July 2, 2009
  4. Committee of Interns and Residents. www.cirseiu.org/assets/Accessed July 2, 2009
  5. Philibert I, Friedmann P, Williams WT. New requirements for resident duty hours (ACGME. Work Group on Resident Duty Hours). JAMA. 2002;288:1112–1114
  6. Accreditation Council for Graduate Medical Education. Innovative Approaches for Resident Duty Hours. http://www.acgme.org/acWebsite/dutyHours/dh_innovative.aspAccessed July 6, 2009
  7. Institute of Medicine. http://www.iom.edu/CMS/3809/48553/60449.aspxAccessed July 6, 2009
  8. Council Directive 93/104/EC. Official Journal of the European Community. 1993;L307:18–24
  9. Denisco RA, Drummond JN, Gravenstein JS. The effect of fatigue on the performance of a simulated anesthetic monitoring task. J Clin Monit. 1987;3:22–24
  10. Barger LK, Cade BE, Ayas NT, et al. Extended works shifts and the risk of motor vehicle crashes among interns. N Engl J Med. 2005;352:125–134
  11. Ayas NT, Barger LK, Cade BE, et al. Extended work duration and the risk of self reported percutaneous injuries in interns. JAMA. 2006;296:1055–1062
  12. Arora VM, Georgitis E, Siddique J, et al. Association of workload of on-call medical interns with on-call sleep duration, shift duration, and participation in educational activities. JAMA. 2008;300:1146–1153
  13. Friesen LD, Vidyarthi AR, Baron RB, Katz PP. Factors associated with intern fatigue. J Gen Intern Med. 2008;23:1981–1986
  14. Fletcher KE, Davis SQ, Underwood W, et al. Systematic review: effects of resident work hours on patient safety. Ann Intern Med. 2004;141:851–857
  15. Landrigan CP, Fahrenkopf AM, Lewin D, et al. Effects of the accreditation council for graduate medical education duty hour limits on sleep, work hours, and safety. Pediatrics. 2008;122:250–258
  16. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform. JAMA. 2007;298:984–992
  17. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among patients in Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. JAMA. 2007;298:975–983
  18. Shetty KD, Battacharya J. Changes in hospital mortality associated with residency work-hour regulations. Ann Intern Med. 2007;147:73–80
  19. Saunders DL, Kehoe KC, Rinehart VH, Berg BW. Self-reporting of internal medicine housestaff work hours. Hawaii Med J. 2005;64:14–16
  20. Accreditation Council for Graduate Medical Education. Internal Medicine Program Requirements. http://www.acgme.org/acWebsite/RRC_140/140_prIndex.aspAccessed July 6, 2009

 Funding: None.

 Conflict of Interest: None of the authors have any conflicts of interest associated with the work presented in this manuscript.

 Authorship: All authors had access to the data and played a role in writing this manuscript.

PII: S0002-9343(09)01064-X

doi:10.1016/j.amjmed.2009.10.009

The American Journal of Medicine
Volume 123, Issue 3 , Pages 286-290, March 2010

Article Tools