If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Brigham and Women's-Faulkner Hospital Academic Hospitalist Service, Boston, MassDivision of General Internal Medicine, Brigham and Women's Hospital, Boston, Mass
Divisions of General and Hospital Internal Medicine, Mayo Clinic, Rochester, MinnOffice of Educational Innovations, Internal Medicine Residency, Mayo Clinic, Rochester, Minn
Divisions of General and Hospital Internal Medicine, Mayo Clinic, Rochester, MinnOffice of Educational Innovations, Internal Medicine Residency, Mayo Clinic, Rochester, Minn
Brigham and Women's-Faulkner Hospital Academic Hospitalist Service, Boston, MassDivision of General Internal Medicine, Brigham and Women's Hospital, Boston, Mass
Requests for reprints should be addressed to LeRoi S. Hicks, MD, MPH, Division of Hospital Medicine, UMass Memorial Healthcare, Memorial Campus, 119 Belmont Street, Worcester, MA 01605
Division of Hospital Medicine, UMass Memorial Healthcare, WorcesterDepartment of Quantitative Sciences, University of Massachusetts Medical School, Worcester
The landscape of medical education changed substantially in 2003 when the Accreditation Council for Graduate Medical Education (ACGME) first implemented its resident duty-hour restriction.
Since this implementation, little is known about the amount of continued variation of resident workload between residency programs and the effect of workload on patient outcomes. Several studies have examined the association of the ACGME changes with patient safety outcomes, mortality, quality of care, and specialty care utilization within the hospital. Results of these studies have overall shown improved or equivalent care since the duty-hour implementation,
Primary affiliated hospitals of Internal Medicine residency programs that train their house staff in patient handoffs are associated with decreased pneumonia mortality rates.
•
Variations in resident workload are not associated with differences in quality of care, readmission, or mortality rates for common inpatient diagnoses.
•
Residency programs may find value in increasing efforts at improving care transitions, particularly as further Accreditation Council for Graduate Medical Education restrictions are implemented.
Recently, a few studies have directly examined the effect of resident workload on patient outcomes in the current climate of resident duty-hour restrictions. These studies have demonstrated that increasing numbers of admissions on call days is associated with higher costs, increased length of stay, and higher mortality; however, increased daily workload is associated with lower costs and length of stay, and decreasing patient load improves information transfers at patient discharge.
To our knowledge, no prior studies have examined the impact of resident handoff training or evaluation on patient outcomes. In theory, without adequate training in handoffs, the benefits of well-rested house staff may be offset by the hazards of discontinuity of care.
making it crucial to better understand how workload variation and handoff training affect patient outcomes. In this study, we examined nationally representative data to assess the association among resident workload, transitions in care training, and evaluation on quality of care and outcomes for patients hospitalized in medical and surgical hospitals throughout the US.
Methods
Overview
To assess whether variability in resident workload and training program management of transitions of care are associated with patient outcomes, we examined data from 3 nationally representative datasets: 2007 Hospital Quality Alliance (HQA) data,
Utilizing these data, we examined whether program director reporting of resident workload and handoff training and evaluation were related to quality of care, readmission rates, and mortality for the 3 most common inpatient diagnoses in the US, adjusting for differences in hospital and program characteristics. Our study protocol was approved by the Partners Healthcare Human Subjects Review Committee. Use of the APDIM survey data was approved by the Institutional Review Board of the Mayo Clinic.
Data Sources
Outcomes
Outcomes of interest were obtained from the HQA database and included hospital-level quality-of-care process indicators as well as 30-day risk-adjusted readmission and mortality rates for acute myocardial infarction (AMI), congestive heart failure (CHF), and pneumonia. HQA is distributed by the Center for Medicare and Medicaid Services and reports on various performance metrics of hospitals for patients age 65 years or older who are enrolled in Original Medicare (traditional fee-for-service Medicare) for the entire 12 months before their hospitalization.
The quality process indicators were combined into composite measures by disease state for AMI, CHF, and pneumonia (Table 1). The indicators used for the composite measures were limited to those related to care by internal medicine residents, with indicators chosen by manual review and by prior literature examining hospitalist effects on quality of care.
Eight individual indicators were excluded from the composite measures, for example, “time to thrombolysis of 30 minutes or less” for AMI patients (which is primarily performed in the emergency department before admission).
Table 1Composite Process Measures
Condition
Hospital Quality Measure
Acute myocardial infarction
Given ACE inhibitor for left ventricular systolic dysfunction
Given aspirin at discharge
Given beta-blocker at discharge
Given smoking cessation counseling
Congestive heart failure
Given ACE inhibitor or ARB for left ventricular systolic function
Excluded acute myocardial infarction measures include “Given aspirin on arrival,” “Given fibrinolytic medication within 30 minutes of arrival,” “Given PCI within 30 minutes of arrival.”
Excluded pneumonia measures include “Given initial antibiotics within 6 hours of arrival,” “Given oxygenation assessment,” “Given most appropriate initial antibiotics,” “Initial ER blood cultures drawn prior to administration of antibiotics.”
The composite measures were created by dividing the number of times a hospital performed appropriate care for these indicators by the number of times they were eligible to provide appropriate care for the indicators, a method that has been used in prior research with these data.
Workload and Transitions in Care Training and Evaluation
Four predictors of interest were obtained from the 2008 APDIM Survey, which is an annual survey that tracks characteristics of internal medicine residency programs in the United States. The 2008 survey included questions related to resident work load and patient handoffs. Survey methodology has been published previously, and all questions and basic distributions of responses for the entire survey are publicly available.
In August 2008, an e-mail notification with a link to a Web-based questionnaire was sent to each of the 373 member programs of APDIM, representing 97% of the 383 US categorical internal medicine residency programs.
The 2 resident workload predictors included “mean maximum census,” defined as the reported mean maximum number of patients an intern carries on a daily basis on general medicine rotations; and “mean maximum admits,” defined as the reported mean maximum number of patients an intern admits on call days on general medicine rotations. The 2 predictors related to patient handoffs included residency program director reports of whether programs trained or evaluated their residents on patient handoffs (Table 2).
Upon examination of the workload predictors, a small number of outlier values were noted for “mean maximum census” with a lowest value of 0 and a highest value of 50. To adjust for this range, we excluded the extreme 5% (top and bottom 2.5%) of values for this predictor from our analysis, resulting in a sample of 171 hospitals with final values ranging from 6 to 15 patients. From this pool, we excluded 2 hospitals reporting a “mean maximum admits” of 0 to limit our evaluation to hospitals where house staff admit patients during call days.
Hospital Characteristics
We collected data for each residency program's primary affiliated hospital. Residency program primary affiliated hospitals were identified using the accredited residency program search on the ACGME Website.
Covariates chosen for the analyses included each hospital's geographic location, ownership, urban versus rural setting, nursing intensity (ratio of registered full-time nurses to patient staffing by 1000 patient days), teaching intensity (ratio of full-time residents to total number of patient beds), proportion of Medicare and Medicaid patients by admission, and number of hospital beds (dichotomized into ≤200 vs >200 beds) obtained from the AHA database, and 3-year rolling pass rate for American Board of Internal Medicine certification examination and affiliation of residency program with a cardiology fellowship obtained from the APDIM data. Covariates were chosen as they have previously been used and shown significant in most studies using HQA and AHA datasets.
We limited our analysis to primary affiliated hospitals of the internal medicine residency programs with complete predictor and outcome data.
Baseline characteristics in the final cohort were examined and compared with baseline characteristics of primary affiliated hospitals of all 382 internal medicine residency programs within the US to evaluate possible response bias. To analyze the effect of predictors on the composite process measures, we fitted a series of binomial logistic regressions that estimated the number of times a group of composite indicators were met successfully by the hospital by the number of times the hospital was eligible to meet those indicators. Bivariable analysis was performed, followed by fully adjusted analyses with the addition of the 3 other predictors of interest and all covariates. In secondary analyses, an interaction term for training and evaluation of patient handoff was added to the multivariable model to evaluate for effect modification and was subsequently removed from the final model after it was found to be nonsignificant. Generalized estimating equations (the GENMOD procedure in SAS statistical software, v9.2; IBM, Armonk, NY) were used to account for clustering of scores at the hospital level.
To examine the effect of our predictors on readmission and mortality rates for AMI, CHF, and pneumonia, a series of linear regression models were run to estimate the crude effect of each of the 4 predictors on these outcomes. We then added all other predictors and covariates for the multivariable linear regression analyses to obtain our adjusted effects. As above, effect modification for training and evaluation of patient handoffs was nonsignificant.
Results
Hospital Characteristics
Of the 373 internal medicine residency programs that were sent surveys, 268 (72%) responded with similar program characteristics between responders and nonresponders.
Of these programs, 169 had available predictor and outcome data and were included in final analysis. The average reported mean maximum census was 10.5 patients and average mean maximum admits were 5.4 patients. One hundred nineteen (70%) reported that they trained their house staff in handoffs and 89 (53%) reported that they evaluated their house staff in handoffs. Characteristics among the 169 primary affiliated hospitals included in the final analysis and all primary affiliated hospitals of the 383 internal medicine residency programs did not vary greatly (Table 3).
Table 3Baseline Characteristics of All Primary Affiliated Hospitals of Internal Medicine Residency Programs and Final Cohort Analyzed
Predictor variables in Initial Cohort equal to Final Cohort as final cohort chosen due to available predictor data. Missing data for listed characteristics ranged from 7 to 39 in Initial Cohort (no missing data in Final Cohort).
Teaching intensity defined as ratio of full-time residents to total number of patient beds.
0.35 (0.36)
0.36 (0.36)
Proportion of Medicare patients, mean (SD)
0.36 (0.11)
0.37 (0.10)
Proportion of Medicaid patients, mean (SD)
0.24 (0.14)
0.22 (0.12)
Hospital size, n (%)
≤200
33 (9)
13 (8)
>200
310 (91)
156 (92)
ABIM rolling pass rate 2006-2008, mean (SD)
92.4 (6.1)
92.1 (6.3)
Programs affiliated with Cardiology fellowship, n (%)
124 (32)
78 (46)
ABIM=American Board of Internal Medicine.
Predictor variables in Initial Cohort equal to Final Cohort as final cohort chosen due to available predictor data. Missing data for listed characteristics ranged from 7 to 39 in Initial Cohort (no missing data in Final Cohort).
† Nursing intensity defined as ratio of registered full-time nursing to patient staffing by 1000 patient days.
‡ Teaching intensity defined as ratio of full-time residents to total number of patient beds.
In unadjusted analyses, each 1-patient increase in the mean maximum census was associated with 14% lower odds of meeting composite quality measures for pneumonia (P=.005) and 7% lower odds for meeting composite quality measures for AMI (P=.05) and CHF (P=.05), although these associations became nonsignificant with multivariate adjustment. Mean maximum admits were not statistically significantly associated with odds of meeting composite process measures for any disease state.
Unadjusted analyses of the effect of resident workload on readmission rates demonstrated that for every increase in mean maximum census by 1 patient, the absolute 30-day risk-adjusted readmission rate for AMI increased by 0.16% (P=.04), and the readmission rate for CHF increased by 0.23% (P=.04). However, these effects lost significance upon adjustment. There were no effects of resident workload on readmission rates for pneumonia or mortality rates for any condition.
Training and Evaluation in Handoffs
There were no significant associations between handoff training and evaluation on composite quality measures for AMI, CHF, and pneumonia in either the unadjusted or adjusted analyses. Similarly, there were no significant effects of handoff training or evaluation on readmission rates for AMI, CHF, or pneumonia.
Handoff training and evaluation also had no significant association with mortality rates for AMI or CHF. However, programs that evaluated their residents on patient handoffs had a significantly increased pneumonia mortality rate (11.6% vs 10.8%, P=.02), although this association became nonsignificant with adjustment (P=.07). Last, programs that trained their residents on handoffs had a significantly decreased unadjusted and adjusted pneumonia mortality rate (adjusted 11.0% vs 11.8%, P=.01) compared with programs that did not (Figure).
FigureAdjusted effects of handoff training and evaluation on 30-day risk-adjusted mortality rates for pneumonia.
In this nationally representative evaluation of internal medicine residency-affiliated primary hospitals, we found that resident workload had no significant association with quality of care and patient outcomes for the most common inpatient diagnoses. We did note that primary affiliated hospitals of internal medicine residency programs that train their residents in patient handoffs have lower 30-day risk-adjusted mortality rates for patients with pneumonia, controlling for hospital and program characteristics. To our knowledge, this is the first study to demonstrate the effects of resident workload and residency handoff training and evaluation on patient outcomes on a national level.
The lack of association between resident workload and patient outcomes contradicts some prior studies. Those reports examining the effect of the implementation of duty-hour restrictions by ACGME in 2003 have shown mixed results; some studies have demonstrated no significant differences in mortality for patients with AMI, CHF, or pneumonia,
However, these studies were performed immediately following the duty-hour restriction implementation of 2003 and therefore, are due for more current examination. Furthermore, they solely examined the effect of the duty-hour restrictions as opposed to specifically looking at resident workload; most studies were site-specific. Fewer recent studies that look directly at the effect of resident workload on patient outcomes have demonstrated overall improved outcomes with decreased workload. Coit et al
demonstrated that decreased resident daily census resulted in improved quality of discharge summaries as a surrogate for resident performance, and Ong et al
demonstrated that increased call day admissions were associated with increased length of stay, cost, and risk of inpatient mortality. However, this body of literature is limited and site-specific, and therefore our nationally representative study adds to these data.
We also found an association between handoff training and lower pneumonia mortality rates. As a consequence of duty-hour restrictions, residents participate in more patient handoffs.
There are currently limited data examining the effects of transitions of care within a patient's hospitalization. Existing studies suggest that poor handoffs lead to worse patient outcomes, including adverse events,
Furthermore, surveys administered to surgeons, hospitalists, and residents demonstrate that health care providers estimate that 15% to 70% of medical errors are attributable to communication breakdown or inadequate handoffs.
The intent of handoff training is to improve transfer of information from one provider to another. Given that more complex patients may be more susceptible to adverse events in the case of poorer handoffs, handoff training may differentially impact older and more medically complex patients. As age and comorbid conditions are the 2 factors most highly associated with pneumonia mortality,
improving the communication process among providers may result in improved pneumonia mortality rates. Although similar vulnerability exists among patients with AMI and CHF, it is possible that among the hospitals examined, these patient groups are more affected by house staff on the cardiology service as opposed to the general medicine service.
Our findings are subject to several limitations. First, our predictors of interest were obtained from survey data, which is subject to recall bias and lacks qualitative information. Recall bias may particularly be present in the resident workload predictors, as program directors were required to estimate intern workload throughout the year. In an attempt to adjust for this bias, we omitted the outlier data for these predictors. This bias is much less likely for the handoff predictors, given the “yes/no” aspect of the survey questions. If program directors confused the terms “training” and “evaluation” in handoffs, one might expect an inverse relationship in the reporting of these 2 predictors, but we did not find such an effect. However, qualitative information on handoff training and evaluation was lacking. Second, our analyses measured hospital-level data and predictors were at the level of the residency program. Thus, it is possible that we measured the effects of environmental characteristics other than the house staff training in handoffs on patient outcomes for pneumonia. To address this limitation, we repeated our analysis stratified by different levels of teaching intensity (resident-to-bed ratio) and found a consistent effect of handoff training on pneumonia mortality regardless of hospital strata. Third, although we examined quality-of-care outcomes that are standard in research and prespecified our predictors and outcomes a priori, we examined our outcomes across 3 different conditions, and therefore the level of statistical significance of our findings should be interpreted with caution. Lastly, given the cross-sectional study design, we cannot prove causality of our associations.
In summary, our study of a national sample of internal medicine residency-affiliated hospitals demonstrated that programs that train house staff in handoffs may have significantly better outcomes for pneumonia patients than those that do not. The relationship between residency training within teaching hospitals and patient handoffs is particularly important in today's climate of further duty-hour restrictions necessitating increased handoffs.
Given our findings, residency programs may find value in increasing efforts at improving care transitions. Future studies should examine what mediates the association between handoff training and improved pneumonia outcomes, particularly qualitative information on handoff training and impact on patient care.
Acknowledgements
We greatly acknowledge the efforts of the Mayo Clinic Survey Research Center for their assistance with design and data collection of the Association of Program Directors of Internal Medicine survey. This study was supported in part by the Mayo Clinic Internal Medicine Residency Office of Educational Innovations. Dr Mueller was also supported by an Institutional National Research Service Award #T32-HP10251 and by the Division of General Medicine at Brigham and Women’s Hospital.
References
Accreditation Council for Graduate Medical Education (ACGME)
Conflict of Interest: Dr LeRoi Hicks is on the Board of Directors of Health Resources in Action and a scientific advisor to the Health Management Corporation. All other authors declare that they have no conflicts of interest with regard to this manuscript.
Authorship: This manuscript represents original work, and has been approved by all authors, each of whom contributed significantly to this manuscript.
Formatting of questions from the Association of Program Directors of Internal Medicine (APDIM) survey.
00832-1&id=gr1.jpg){kind=link}