The American Journal of Medicine
Volume 116, Issue 3 , Pages 188-193, 1 February 2004

Leapfrog and critical care: evidence- and reality-based intensive care for the 21st century

  • Constantine A Manthous, MD

      Affiliations

    • Pulmonary and Critical Care, Bridgeport Hospital, Bridgeport, Connecticut, and Yale University School of Medicine, New Haven, Connecticut, USA
    • Corresponding Author InformationRequests for reprints should be addressed to Constantine A. Manthous, MD, Bridgeport Hospital, 267 Grant Street, Bridgeport, Connecticut 06610, USA

Received 22 November 2002; accepted 18 August 2003.

Article Outline

Abstract 

In 2000, the Business Roundtable published its Leapfrog report, which contained suggestions for improving administration of critical care. The Leapfrog Group intends to influence health care policy by pressuring insurers and hospitals to implement its guidelines, and both internists and intensivists are likely to be affected if these recommendations are realized. This article outlines the Leapfrog standards for critical care and examines critically the evidence used to justify them. Aside from the guideline that all critically ill patients should be cared for by intensivists, Leapfrog's standards for critical care are based either on weak or no scientific evidence. Rather, most of the guidelines are grounded in common sense and rational extrapolation of the data; as such, they are a reasonable starting point for debate by physicians and policymakers about optimal methods of achieving intensivist-guided care of critically ill patients.

 

The Leapfrog Group, comprised “of more than 130 public and private organizations that provide health care benefits” under the auspices of the Business Roundtable, met in the late 1990s to define standards for hospital care. The resulting publications (1,2) included specific suggestions for improvement of critical care. Hospital administrators are anxious to become “Leapfrog compliant,” fearing that failure to do so will threaten contracts with insurers. If implemented, Leapfrog-driven changes are likely to affect the practice of critical care in the United States. This paper reviews the Leapfrog standards for critical care, illuminates the quality of evidence for each, and examines impediments to implementation.

Back to Article Outline

The original leapfrog guidelines 

The following are the primary recommendations of the Leapfrog Group regarding critical care, which were issued in November 2000 1, 2:

1. Intensive care units (ICUs) should be staffed by Board-certified intensivists, to coordinate and manage care of patients.

2. Intensivists should staff ICUs during daytime hours, a minimum of 8 hours, 7 days per week.

3. Intensivists should respond to more than 95% of calls for assistance within 5 minutes.

4. The intensivist, a “fundamentals of critical care”–certified physician or “physician extender” (also described in some Leapfrog documents as “effectors”), should arrive at the bedside within 5 minutes in 95% of cases.

Certification of intensivists is achieved by completion of an accredited fellowship program sanctioned by the Boards of Internal Medicine, Surgery, Pediatrics, and Anesthesiology. Certification to become an “extender” is achieved by attending a 2-day course sponsored by the Society of Critical Care Medicine. Care “extenders” are physicians or allied health care personnel who provide critical care during hours when the intensivist is not available on site.

Back to Article Outline

Evidence for each of the leapfrog guidelines 

Eight studies 3, 4, 5, 6, 7, 8, 9, 10 were cited to justify the 2000 Leapfrog guidelines (Table 1). Understanding the designs and specific interventions of each study is critical in determining the strength of the evidence that supports Leapfrog's suggestions.

Table 1. Studies Cited in Leapfrog (2000) to Support Recommendations for Physician Staffing in the Intensive Care Unit*
First Author (Reference)Study DesignDuration (years)No. of PatientsICU TypeComparisonOutcomes
Reynolds (3)Retrospective2212MedicineOpen vs. closedHospital mortality decreased from 74% to 57%; no differences in other outcomes.
Brown (4)Retrospective2439Medicine-surgeryOpen vs. intensivist-guided openICU mortality decreased from 27.8% to 13.4%; hospital mortality decreased from 35.5% to 24.5%.
Carson (5)Prospective0.33245MedicineOpen vs. closedNo absolute reduced mortality, hospital stay, or resource utilization.
Manthous (6)Retrospective2930MedicineOpen vs. intensivist-guided openHospital mortality decreased from 34% to 25%; hospital stay was reduced by 5 days; improved house staff knowledge.
Multz (7)Retrospective and prospective0.5586MedicineOpen vs. closedNo effect on mortality; shorter ICU (12.6 vs. 6.1 days) and hospital (31.2 vs. 22.2 days) stays.
Ghorra (8)Retrospective1274SurgeryOpen vs. closedHospital mortality decreased from 14.4% to 6.0%; complications decreased from 55.8% to 44.1; resource utilization and lengths of stay were similar.
Hanson (9)Prospective1198SurgeryOpen vs. intensivist-guided openNo effect on mortality; fewer complications (0.5 vs. 1.7 per patient); shorter ICU (2 vs. 2.8 days) and hospital (20.3 vs. 23.6 days) stays; lower cost ($34,500 vs. $47,500).
Pronovost (10)Retrospective32987SurgeryDaily rounds by intensivist vs. rounds without intensivistDaily rounds by an intensivist was associated with threefold lower mortality and reduced length of stay.

ICU = intensive care unit.

* Open ICUs are those in which patients may be admitted under the primary care of their primary physician, with or without the consultation of an intensivist. Closed ICUs are those in which patients are admitted to the service of an intensivist who directs all care. In most of the comparisons, an open model connotes that, unless specified, no intensivist consultation was stipulated.

1. Intensivists should manage care in the ICUs 

Effect on mortality 

The available data suggest that mortality is reduced when intensivists are involved in patient management. However, none of the three prospective studies cited to formulate the initial Leapfrog guidelines (two performed in medical ICUs and one in a surgical ICU) demonstrated an effect on unadjusted mortality 5, 7, 9. The other retrospective studies assessed several different intensivist-related interventions (Table 1), and are not readily analyzed by meta-analytical techniques (11). Thus, it is not appropriate to group the available studies together, since the types of ICUs (e.g., open vs. closed, university vs. community) and interventions differed substantially among reports. For example, our study (6) examined the effects of adding a full-time intensivist in a teaching, community hospital's medical ICU, maintaining an open model of care. (In an open ICU, all primary care physicians have admission privileges and the right to consult whomever they wish. Consultation with an intensivist may or may not be required in an open system.) In our study, the intensivist was on service 6 months of the year, primarily responsible for or consulted on a minority of patients, and was responsible for several changes in ICU organization. In contrast, Carson and colleagues (5) examined the effects of changing from an open system, in which internists admitted patients with daily intensivist consultation on all patients, to a closed system, in which intensivists assumed primary responsibility for all patients for the duration of ICU stay. That study was performed in a university medical ICU; the same group of intensivists were involved during both study periods. Differences in the specific interventions and milieus in these two studies were likely to have affected the outcomes of the studies.

Very few studies listed in Table 1 were performed in the same environments with the same interventions. Leapfrog authors acknowledged this problem: “There was substantial variation in the ‘intervention’… some interventions involved simply adding co-management by a single intensivist to a system primarily run by non-ICU physicians; others described extensive changes in staff organization” (2). Accordingly, they used what they believed to be a “conservative” estimate, that implementing ICU physician staffing would reduce mortality by 15% when they estimated “approximately 53,850 lives could be saved” if all ICUs in the country were staffed by intensivists (2). Note, however, that the most conservative estimate was that there is no statistically significant reduction in mortality.

A meta-analysis that was published after the guidelines were issued examined the available evidence up to 2001 (12). Ignoring for a moment whether meta-analysis is appropriate for such disparate study designs, the report concluded that a preponderance of data demonstrated that intensivists improve outcomes. However, there appears to be at least one error in the meta-analysis that led to overestimation of the mortality benefit of high-intensity critical care as it was defined. Our study (6) was listed with the third highest weight of evidence for hospital mortality, and was considered as having a high-intensity arm. However, the definition of high intensity was not met during the intervention of our study. Our ICU remained open, a mandatory intensivist consultation was not mandated, and the intensivist was only on service 6 months of the year. (Thus, our study included “intensivist-driven care,” but not “high intensity” care.) Another possible weakness of this meta-analysis was that five of the 13 studies that demonstrated reduced ICU mortality associated with high-intensity care were cited as abstracts. Although no published studies have demonstrated harm associated with intensivist care, the strength of the relation of intensivist staffing and improved mortality in this oft-cited analysis may have been overstated.

Length of stay and cost 

The preponderance of evidence indicates that lengths of stay are reduced (or, at worst, no different) when intensivists are involved in the care of critically ill patients than when intensivists are not involved (Table 1) (12). However, it is not known whether intensivists reduce overall costs of care. Intensivists have been successful at implementing evidence-based protocols. For example, Smyrnios and colleagues (13) reduced length of stay and associated costs (by almost $30,000 per patient) by implementing a weaning protocol for patients who were being ventilated mechanically. They also demonstrated improved hospital throughput and a trend toward improved mortality. This was not as a result of adding an intensivist, but from implementing evidence-based guidelines in a single area of patient management.

It is unclear whether the actual costs of providing round-the-clock intensivists are offset by actual reductions in length of stay, improved hospital throughput, and more efficient use of tests and procedures. One analysis projected that implementation of Leapfrog guidelines in a formerly noncompliant six-bed ICU might save the average hospital $770,000 each year ($2.1 million per year for a 12-bed ICU and $3.4 million per year for an 18-bed ICU) (14). These authors warned, however, that “true costs would be expected to vary widely across hospitals” and “savings estimates are also imprecise and assume that published descriptions of ICU physician staffing effectiveness can be generalized to other settings.”

2. Intensivists should staff ICUs during daytime hours, a minimum of 8 hours 

There are insufficient data to demonstrate that ICUs with on-site intensivists working a minimum of 8 hours per day have better outcomes than other ICUs. Leapfrog sought to address this deficiency by a post hoc telephone poll of study authors about the number of hours intensivists spent in ICUs in various studies (15). The authors listed the number of hours intensivists spent in ICUs, but did not examine whether those with 8-hour, 7-day coverage had better outcomes than those with less coverage. Nonetheless, the working group concluded that they “found evidence to support the components of the Leapfrog ICU physician staffing,” including adherence to the 8-hour rule for 7 days per week.

Examination of the data 12, 15 indicates that unadjusted, in-hospital mortality was reduced in only one of the four studies that reported ≥8 hours of weekend intensivist presence, compared with reduced mortality in seven of the eight studies in which intensivists spent less than 8 hours in the hospital on weekends. A prospective study is needed to determine whether patients cared for by intensivists 8 hours per day, 7 days per week, have better outcomes than those with less on-site coverage.

3. Intensivists should respond to more than 95% of calls for assistance within 5 minutes 

No study stated explicitly that intensivists responded to 95% of calls within 5 minutes. However, in the Leapfrog update (15), the authors reported that in every study (in which they could contact authors), pages were returned within 5 minutes, 95% of the time. Again, these studies were not designed a priori to test whether changes in the call-back time affected outcomes. Also, because all studies met this criterion, it is not possible to determine whether it contributed to improved outcomes.

4. Intensivists or “extenders” should arrive at the bedside within 5 minutes in 95% of cases 

There are no data to substantiate this standard, nor are there any data to demonstrate that the Society of Critical Care extender certification course achieves the desired outcomes. Since all published studies were performed before extender certification became available, its effect on outcomes is unknown. It is highly likely that in the majority of cited studies, which were performed predominantly in teaching institutions, residents (not certified as intensivist-extenders) responded to the bedside within 5 minutes and contacted intensivists when appropriate. However, the hypothesis that 5-minute bedside response affects outcomes has not been tested prospectively. In the published update of ICU staffing standards, Leapfrog's medical consultants listed this recommendation but presented no data to substantiate its validity (15).

Back to Article Outline

Additional leapfrog guidelines for critical care in 2003 

In addition to maintaining the previous guidelines, the medical advisory panel of the Leapfrog Group suggested the following additional measures (15):

1. Expand Leapfrog standards to pediatric critical care.

2. Loosening of criteria for qualifications of “intensivists” (to “grandfather” practitioners who graduated before 1987).

3. Limit care of intensivists to one ICU at any given time.

Except for extension of guidelines to pediatric intensive care, for which there is a limited amount of data (16), the amendments are not supported by results from published studies.

Back to Article Outline

Feasibility of implementing Leapfrog ICU staffing guidelines 

At present, there are insufficient numbers of practicing intensivists to staff all hospitals to meet Leapfrog standards (17). A study by the Committee on Manpower for Pulmonary and Critical Care Societies indicated that intensivists provided care to 37% of critically ill patients in 1997, and projected that “demand will grow rapidly while supply will remain near constant, yielding a shortfall of specialist hours equal to 22% of demand by 2020 and 35% by 2030, primarily because of the aging of the US population” (17). In its most recent publications, Leapfrog consultants acknowledge the problem of insufficient manpower 14, 15.

An amendment that is intended to address the manpower shortage can only be found in the Leapfrog Question-Answer documents 18, 19. It suggests that intensivists' care provided by telemedicine will satisfy Leapfrog guidelines if implemented properly (Table 2) 18, 19. A single study reported that institution of telemedicine critical care was associated with reduced in-hospital mortality, using a nonrandomized pre-post design (20). However, to date, this relatively small study remains the only published investigation of the potential effectiveness of ICU telemedicine, and the possible effects on mortality (14 deaths in the 202 patients in the most similar baseline period vs. 9 deaths in the 201 patients after telemedicine was implemented) do not appear to be statistically significant, as was claimed in the original report. Additional studies are required to address the efficacy of ICU telemedicine.

Table 2. Leapfrog Criteria for Telemedicine
1.An on-site intensivist collaborates with a remote (tele-)intensivist, sharing all pertinent information.
2.A tele-intensivist is available for all hours when the on-site intensivist is not available.
3.The tele-intensivist has immediate access to patient data.
4.Data links are available >98% of the time and are secure, to maintain patient confidentiality.
5.Tele-intensivists are able to visualize the patient through clear audiovisual transmissions.
6.Written standards of remote care include that the tele-intensivist is Critical Care Board certified, licensed to practice in the state of on-site care, is credentialed in the on-site location, and remains under the hospital's quality assurance purview; that explicit policies are available to guide roles and responsibilities; and that staff members are educated as to these roles and responsibilities.
7.Tele-intensivists review patients at a frequency that is appropriate for their acuity of illness.
8.Tele-intensivists can respond to provide complete assessment of problems within 5 minutes of any request for assistance.
9.There are written rules to guide effective communication of the tele-intensivist and on-site personnel.
10.Tele-intensivists provide written documentation of their assessments and suggested interventions.

From reference 18.

The manpower shortage will be attenuated, albeit minimally, by recent increases in the number of training positions in critical care medicine (21); larger increases will be necessary. The Leapfrog Group's recent Question-Answer publication suggests that physicians who are Board certified in emergency medicine and who have completed a fellowship in critical care medicine (a subspecialty certification not offered by the American Board of Emergency Medicine) should be eligible to practice intensive care. Certainly, there are no data to support this suggestion.

Back to Article Outline

Implications of noncompliance 

It is not known whether, or to what degree, institutions that are unwilling or unable to comply with the Leapfrog guidelines will be harmed. If insurers, however, adopt Leapfrog criteria, pressure to comply may become considerable, thereby creating a critical manpower shortage. Is it fair for Leapfrog-noncompliant institutions to be excluded from insurance contracts when there are insufficient numbers of intensivists to satisfy Leapfrog demands, or to penalize institutions for noncompliance with the guidelines when they are not evidence based? Should hospitals be denied remuneration, further compromising ability to provide high-quality critical care services if they have the resources to meet some, but not all, of the guidelines? Indeed, stringent application of Leapfrog guidelines could do more harm than good at many hospitals.

Back to Article Outline

Conclusion 

Many of the Leapfrog Group’s standards for critical care are not grounded sufficiently in evidence to mandate their stringent and universal implementation. Although outcomes of critically ill patients are better when their care is managed directly or with the contributions of intensivists (12), implementation of this single goal will face substantial obstacles. The remaining Leapfrog guidelines currently lack the scientific evidence that should be required before mandating national compliance. These guidelines make common sense, however, and if sufficient manpower were available, they could serve as the starting point to formulate realistic goals.

Many additional questions remain to be answered. What is the role of the emerging hospitalist movement in addressing the critical care manpower shortfall? Both hospitalists and intensivists are hospital-based physicians with overlapping “turf” and a common goal, and a natural marriage may emerge 22, 23. Certainly, if hospitalists, specially trained physician assistants, or critical care registered nurses obtain certification to become extenders under the supervision of intensivists, fewer intensivists will be necessary. It remains to be determined whether this model of care achieves outcomes that are similar to those of intensivists (12).

The optimal model of ICU practice also remains unclear. There are few, if any, studies that demonstrate that closed ICU care yields superior results to open units in which intensivists and primary care physicians collaborate (6). Closed models have been resisted in many hospitals because of inadequate manpower and because primary care physicians have been reluctant to cede decision making, choice of consultants, and billing to intensivists. These concerns might be attenuated at some hospitals by implementing carefully crafted, open models of intensivist-guided care.

The Business Roundtable has started us down a road that could lead to better critical care. As a result of Leapfrog’s efforts, the American Thoracic Society, the Society of Critical Care Medicine, and the American College of Chest Physicians have formed a task force to develop recommendations for ICU care. Remarkably, the Business Roundtable has seized the agenda, compelling physicians, hospital administrators, and policymakers to consider the evolution of evidence-based, cost-effective critical care in the “new health system for the 21st century” (24).

Back to Article Outline

Acknowledgements 

The National Quality Forum has since accepted Leapfrog's recommendations for critical care without evident reanalysis of the supporting data (http://www.qualityforum.org).

Back to Article Outline

References 

  1. Leapfrog fact sheet: ICU physician staffing [Leapfrog Group website]. Available at: http://www.leapfroggroup.org/FactSheets/ICU_FactSheet.pdf. Accessed July 21, 2003
  2. Birkmeyer JD, Birkmeyer CM, Delbanco SF, et al. Leapfrog safety standards: the potential benefits of universal adoption [Leapfrog Patient Safety Standards website]. Available at: http://leapfroggroup.org/PressEvent/birkmeyer.pdf. Accessed July 21, 2003
  3. Reynolds HN, Haupt MT, Thill-Baharozian MC, Carlson RW. Impact of critical care physician staffing on patients with septic shock in a university hospital medical intensive care unit. JAMA. 1988;260:3446–3450
  4. Brown JJ, Sullivan G. Effect of ICU mortality of a full-time critical care specialist. Chest. 1989;96:127–129
  5. Carson SS, Stocking C, Podsadecki T, et al.  Effects of organizational change in the medical intensive care unit of a teaching hospital. JAMA. 1996;276:322–328
  6. Manthous CA, Amoateng-Adjepong Y, Al-Kharrat T, et al.  Effects of a medical intensivist on patient care in a community teaching hospital. Mayo Clin Proc. 1997;72:391–399
  7. Multz AS, Chalfin DB, Samson IM, et al.  A “closed” medical intensive care unit (MICU) improves resource utilization compared with an “open” MICU. Am J Respir Crit Care Med. 1998;157:1468–1473
  8. Ghorra S, Reinert SE, Cioffi W, Buczko G, Simms HH. Analysis of the effect of conversion from open to closed surgical intensive care unit. Ann Surg. 1999;229:163–171
  9. Hanson CW, Deutschman CS, Anderson HL, et al.  Effects of an organized critical care service on outcomes and resource utilization (a cohort study). Crit Care Med. 1999;27:270–274
  10. Pronovost PJ, Jenckes MW, Dorman T, et al.  Organizational characteristics of intensive care units related to outcomes of abdominal aortic surgery. JAMA. 1999;281:1310–1317
  11. Petitti DB. Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis (Methods for Quantitative Synthesis in Medicine). 2nd ed. New York, New York: Oxford University Press; 2000; 83-84
  12. Pronovost PJ, Angus DC, Dorman T, et al.  Physician staffing patterns and clinical outcomes in critically ill patients. A systematic review. JAMA. 2002;6:2151–2162
  13. Smyrnios NA, Connolly A, Wilson MM, et al.  Effects of a multifaceted, multidisciplinary, hospital-wide quality improvement program on weaning from mechanical ventilation. Crit Care Med. 2002;30:1224–1230
  14. Birkmeyer JD, Birkmeyer CM, Skinner JS. Leapfrog safety standards: economic implications [Leapfrog Group website]. Available at: http://www.leapfroggroup.org/toolkit/LF.Costs.Final.pdf. Accessed July 21, 2003
  15. Rockeymoore MB, Holtzmueller CG, Milstein A, Dorman T, Pronovost PJ. Updating the Leapfrog group intensive care unit physician staffing standard. J Clin Outcomes Manage. 2003;10:31–37
  16. Pollack MM, Katz RW, Ruttimann UE, Getson PR. Improving the outcome and efficiency of critical care (the impact of an intensivist). Crit Care Med. 1988;16:11–17
  17. Angus DC, Kelley MA, Schmitz RJ, et al.  Caring for the critically ill patient. Current and projected workforce requirements for care of the critically ill and patients with pulmonary disease (can we meet the requirements of an aging population?). JAMA. 2000;284:2762–2770
  18. Frequently asked questions about the Leapfrog Group's hospital patient safety survey [Leapfrog Group website]. Available: http://leapfrog.medstat.com/content/FAQ.pdf. Accessed July 21, 2003
  19. The Leapfrog Group's Q and A [Leapfrog Group website]. Available at: http://www.leapfroggroup.org/qna.htm#Q601. Accessed July 22, 2003
  20. Rosenfeld BA, Dorman T, Breslow MJ, et al.  Intensive care unit telemedicine (alternate paradigm for providing continuous intensivist care). Crit Care Med. 2000;28:3925–3931
  21. American Board of Internal Medicine. The national study of graduate education in internal medicine: summary of workforce trends and key findings from the internal medicine survey academic year 2001/2002. Additional information [National Study of Graduate Education in Internal Medicine website] Available at: http://www.nasgim.org. Accessed July 22, 2003
  22. Davis KM, Koch KE, Harvey JK, et al.  Effects of hospitalists on cost, outcomes, and patient satisfaction in a rural health system. Am J Med. 2000;108:621–626
  23. Meltzer D, Manning WG, Morrison J, et al.  Effects of physician experience on costs and outcomes on an academic general medicine service (results of a trial of hospitalists). Ann Intern Med. 2002;137:866–874
  24. Committee on Quality of Health Care in America . Crossing the quality chasm. A new health system for the 21st century. Washington, DC: National Academy Press; 2001;

PII: S0002-9343(03)00659-4

doi:10.1016/j.amjmed.2003.08.032

The American Journal of Medicine
Volume 116, Issue 3 , Pages 188-193, 1 February 2004

Article Tools