AJM online Clinical research study| Volume 129, ISSUE 6, P636.e13-636.e20, June 01, 2016

# Clinical Decision Support Reduces Overuse of Red Blood Cell Transfusions: Interrupted Time Series Analysis

Published:February 09, 2016

## Abstract

### Background

Red blood cell transfusion is the most common procedure in hospitalized patients in the US. Growing evidence suggests that a sizeable percentage of these transfusions are inappropriate, putting patients at significant risk and increasing costs to the health care system.

### Methods

We performed a retrospective quasi-experimental study from November 2008 until November 2014 in a 576-bed tertiary care hospital. The intervention consisted of an interruptive clinical decision support alert shown to a provider when a red blood cell transfusion was ordered in a patient whose most recent hematocrit was ≥21%. We used interrupted time series analysis to determine whether our primary outcome of interest, rate of red blood cell transfusion in patients with hematocrit ≥21% per 100 patient (pt) days, was reduced by the implementation of the clinical decision support tool. The rate of platelet transfusions was used as a nonequivalent dependent control variable.

### Results

A total of 143,000 hospital admissions were included in our analysis. Red blood cell transfusions decreased from 9.4 to 7.8 per 100 pt days after the clinical decision support intervention was implemented. Interrupted time series analysis showed that significant decline of 0.05 (95% confidence interval [CI], 0.03-0.07; P < .001) units of red blood cells transfused per 100 pt days per month was already underway in the preintervention period. This trend accelerated to 0.1 (95% CI, 0.09-0.12; P < .001) units of red blood cells transfused per 100 pt days per month following the implementation of the clinical decision support tool. There was no statistical change in the rate of platelet transfusion resulting from the intervention.

### Conclusions

The implementation of an evidence-based clinical decision support tool was associated with a significant decline in the overuse of red blood cell transfusion. We believe this intervention could be easily replicated in other hospitals using commercial electronic health records and a similar reduction in overuse of red blood cell transfusions achieved.

## Keywords

Clinical Significance
• A clinical decision support tool implemented in a commercial electronic health record resulted in a statistically significant decline in the use of inappropriate red blood cell transfusions.
• Reduction in use of inappropriate red blood cell transfusions improves patient safety and reduces health care costs.
• Interrupted time series analysis is a robust method to analyze quasi-experimental studies, and use of a nonequivalent dependent control variable can further strengthen validity.
Red blood cell transfusion is the most frequently used procedure in hospitalized patients in the US, occurring in approximately 8% of all hospital admissions.
• Pfuntner A.
• Wier L.
• Stocks C.
Most Frequent Procedures Performed in U.S. Hospitals, 2011.
The average cost to hospitals to obtain one unit of red blood cells is $225, and with some 1.35 million units transfused annually, estimated costs for acquisition are more than$300 million.
Department of Health and Human Services
The 2011 National Blood Collection and Utilization Survey Report.
While poorly quantified, the costs, direct and indirect, related to storage, testing, and administration likely exceed acquisition costs. Shander et al
• Shander A.
• Hofmann A.
• Ozawa S.
• Theusinger O.M.
• Gombotz H.
• Spahn D.R.
Activity-based costs of blood transfusions in surgical patients at four hospitals.
estimated these costs between $500 and$1200 per unit, costing the health care system an additional $675 million to$1.6 billion yearly. Red blood cell transfusion is not without its risks, including transmission of infectious disease, immunomodulation, and transfusion-related effects.
• Perkins H.A.
• Busch M.P.
Transfusion-associated infections: 50 years of relentless challenges and remarkable progress.
• Carson J.L.
• Grossman B.J.
• Kleinman S.
• et al.
Red blood cell transfusion: a clinical practice guideline from the AABB*.
Over the last decade, significant effort has been undertaken to examine the effectiveness and harms associated with red blood cell transfusion for hospitalized patients with anemia. Much of the effort has been focused on appropriate transfusion strategies, that is, liberal, transfusing red blood cells for a hemoglobin of <9-10 g/dL, vs restrictive, hemoglobin <7-8 g/dL. Hebert et al found that a restrictive transfusion strategy in critical care patients had a potential mortality benefit without significant harm.
• Hebert P.C.
• Wells G.
• Blajchman M.A.
• et al.
A multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. Transfusion requirements in critical care investigators, Canadian Critical Care Trials Group.
Additional studies have examined transfusion strategies in several other patient populations including cardiac surgery, high-risk hip fracture patients, upper gastrointestinal bleeding, and septic shock patients, and again found that restrictive strategies were at least as effective when compared with liberal transfusion strategies.
• Hajjar L.A.
• Vincent J.L.
• Galas F.R.
• et al.
Transfusion requirements after cardiac surgery: the TRACS randomized controlled trial.
• Carson J.L.
• Terrin M.L.
• Noveck H.
• et al.
Liberal or restrictive transfusion in high-risk patients after hip surgery.
• Villanueva C.
• Colomo A.
• Bosch A.
• et al.
Transfusion strategies for acute upper gastrointestinal bleeding.
• Holst L.B.
• Haase N.
• Wetterslev J.
• et al.
Lower versus higher hemoglobin threshold for transfusion in septic shock.
• Murphy G.J.
• Pike K.
• Rogers C.A.
• et al.
Liberal or restrictive transfusion after cardiac surgery.
• Carson J.L.
• Sieber F.
• Cook D.R.
• et al.
Liberal versus restrictive blood transfusion strategy: 3-year survival and cause of death results from the FOCUS randomised controlled trial.
Given these results, wide efforts have been made to encourage restrictive red blood cell transfusion protocols in many clinical settings.
• Carson J.L.
• Grossman B.J.
• Kleinman S.
• et al.
Red blood cell transfusion: a clinical practice guideline from the AABB*.
The Joint Commission
Proceedings from the National Summit on Overuse.
For instance, when the American Board of Internal Medicine's Choosing Wisely campaign elicited key targets to reduce unneeded treatment, the American Society of Hematology and the Society for Hospital Medicine chose reduction of red blood cell transfusion as one of their 5 targets for improvement. While this effort is laudable, physician education alone has often shown modest effect on practice improvement.
• Davis D.A.
• Thomson M.A.
• Oxman A.D.
• Haynes R.B.
Changing physician performance. A systematic review of the effect of continuing medical education strategies.
Other techniques to change provider behavior at the point of care are more effective. Use of clinical decision support through electronic systems to improve adherence to guidelines is well supported by multiple studies.
• Roshanov P.S.
• Fernandes N.
• Wilczynski J.M.
• et al.
Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials.
However, many of the studies supporting clinical decision support tools have come from home-grown (ie, non-commercial) electronic health records (EHR) systems and have occurred in carefully controlled trial-like settings. Comparatively few data exist on implementation of clinical decision support tools in commercial EHRs in routine clinical care.
• Roshanov P.S.
• Fernandes N.
• Wilczynski J.M.
• et al.
Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials.
Expanding use of clinical decision support systems to reduce inappropriate red blood cell transfusion was identified by stakeholders as a top priority; however, the evidence for its efficacy remains lacking.
The Joint Commission
Proceedings from the National Summit on Overuse.
This observational study seeks to examine whether the implementation of a clinical decision support tool can reduce overuse of red blood cell transfusions in hospitalized patients.

## Methods

### Study Setting

This study was conducted at Oregon Health and Sciences University, a 576-bed tertiary care facility in Portland.

### Patient Population

All adult (≥18 years old) patients admitted to all services except Obstetrics, from November of 2008 to November of 2014, were included in the analysis.

### Institutional Review

This study was approved by the local institutional review board and a waiver of informed consent was obtained.

### Study Design

This was a quasi-experimental study with an interrupted time series analysis design. Confounders were accounted for by monitoring platelet transfusion rates during the primary intervention period as a nonequivalent dependent control variable.

### Clinical Decision Support Intervention

The clinical decision support tool for red blood cell transfusion was implemented to improve routine clinical care with no a priori plan for study. It was implemented in October of 2011. At that time it was implemented to all adult services except general surgery and the bone marrow transplant unit. It was further implemented on those units in August of 2013. The EHR used at Oregon Health and Sciences University is Epic (Epic Systems, Verona, WI). The intervention functions in the following manner: when a provider is caring for a patient whose most recent blood hematocrit value is ≥21% and an order for red blood cell transfusion is placed, an interruptive alert is displayed (Supplementary Figure, available online). The alert also allows the user to turn off the alert with common reasons for red blood cell transfusions in patients with hematocrit ≥21%, such as tachycardia, hypotension, active bleeding, acute coronary syndrome, instability, and imminent surgery. In addition to the clinical decision support intervention, ongoing efforts with provider education related to appropriate transfusion use occurred in an ad hoc manner, with a total of 6 department-level talks given over a 2-year period from August of 2010 to August of 2012.

### Outcome Measure

We analyze the use of red blood cell transfusions as a rate. We performed this both with a broad measure using all red blood cell transfusions as the numerator and 100 patient (pt) days as the denominator, as well as with a strict measure using red blood cell transfusions in patients with hematocrit ≥21% as the numerator and using the same denominator.

### Data Collection

Transfusion data, lab values, and patient characteristics were extracted from the clinical database by SZK using Structured Query Language. Briefly, the number of respective transfusions for each blood product type per month was calculated. These results were divided by the total patient days at risk for that respective month to create a rate.

### Data Analysis

Comparisons between the pre- and post-intervention groups for the red blood cell transfusion clinical decision support tool were made with t tests for continuous variables and chi-squared for categorical variables, when appropriate. We performed an interrupted time series analysis using STATA version 13 (StataCorp, College Station, TX).
• Linden A.
Stata module for conducting interrupted time series analysis for single and multiple groups.
We used a 2-sided alpha of 0.05. We tested for auto-correlation across a 12-month period. For the interrupted time series analysis we dealt with missing hematocrit data in the following manner. Firstly, the analysis was performed treating all transfusions that were missing hematocrit values as if they had occurred in patients with hematocrit ≥21 and then repeated the analysis, treating them as if they had occurred in the hematocrit <21 group.
We utilized the rates of platelet transfusion to serve as a nonequivalent dependent control variable from November 2008 until July 2013, at which time a separate clinical decision support tool for inappropriate platelet transfusion was implemented, therefore rendering its use in this fashion obsolete.
• Cook T.
• Campbell D.
Experimental and Quasi-Experimental Designs.
We allowed a 1-month wash-in period and we also performed a sensitivity analysis using a 2-month wash-in period.

## Results

There were approximately equal numbers of admissions prior to and after the implementation of the red blood cell clinical decision support intervention (Table 1). The average age of the patients was older in the postintervention period and there was a slight decrease in the percentage (%) of patients that were female (Table 1). Both the complexity of the patients, as judged from the case mix index, and the mortality were unchanged between the 2 periods. The length of stay was noted to decrease from the pre- to the postintervention period (Table 1).
Table 1Pre- and Postperiod Inpatient Characteristics
CharacteristicPrior to CDSAfter CDSP-Value
Total inpatient days361,686353,439
Mean age (y)53.154.3<.001
Sex (% female)47.646.9.01
Mean case mix index2.012.04.1
Mortality (%)2.02.0.6
Mean length of stay (d)5.054.96.02
Statistical comparisons made with t test for continuous and chi-squared for continuous and categorical variables, respectively.
CDS = clinical decision support intervention.
The probability of an individual patient receiving a potentially inappropriate red blood cell transfusion (ie, when hematocrit ≥21%) was approximately 1.4% lower in the postintervention period, P < .001 (Table 2). Examining the outcome as a rate we found the rate of red blood cell transfusion per 100 pt days was approximately 1.6 (95% confidence interval [CI], 1.2-2.1; P < .001) red blood cell units lower in the postintervention period, or 5645 fewer transfusions during the 3-year post period (Table 2). The entirety of this effect was seen in red blood cells transfused to patients with hematocrit ≥21% per 100 pt days, where there were 2.3 (95% CI, 1.8-2.8) fewer red blood cell units transfused in the postintervention period (Table 2); globally, this was a reduction of 8128 potentially inappropriate red blood cell transfusions.
Table 2Comparison of Red Blood Cell Transfusions in Pre-/Postintervention Periods
ComparisonPrePostDifference (95% CI)P-Value
Probability of receiving red blood transfusion10.0%8.6%<.001
Red blood cell transfusions per 100 patient days9.47.81.6 (1.2-2.1)<.001
Blood transfusions (hematocrit >21%) per 100 patient days6.74.42.3 (1.8-2.8)<.001
Statistical comparisons made with t test for continuous and chi-squared for continuous and categorical variables, respectively.
CI = confidence interval.
The rate of all red blood cell transfusions per 100 pt days was already decreasing by 0.04 (95% CI, 0.07-0.01; P = .005) units per 100 pt days per month in the preintervention period (Figure 1). The rate of red blood cell transfusions continued to decrease by 0.06 (95% CI, 0.08-0.05; P < .001) units per month in the postintervention period (Figure 1). There was no appreciable effect from the clinical decision support tool implementation in terms of the immediate level (ie, intercept), P = .5, or the rate of change (ie, slope), P = .3 (Figure 1).
Using the number of red blood cell transfusions in patients with hematocrit ≥21% per 100 pt days, we found that the rate of use was decreasing in the preintervention period at 0.05 (95% CI, 0.03-0.07; P < .001) units per 100 pt days per month (Figure 1). Examining the postintervention period, we find that the rate of decline is 0.1 (95% CI, 0.09-0.12; P < .001) units per 100 pt days per month. Utilizing interrupted time series analysis, we determined the difference in the rates pre- and postintervention to show a further decrease in the rate of decline at 0.06 (95% CI, 0.03-0.08) units per 100 pt days per month (Figure 2) from the pre- to the postintervention period. Of note, we found a decrease in the variance of the rate of red blood cell transfusion when the clinical decision support tool was implemented across additional patient units, general surgery, and bone marrow transplant, in August of 2013 (Figure 2).
Less than 1% (836 of 84,518) of the transfused units of red blood cells had missing hematocrit value in the 24 hours prior to transfusion administration. The sensitivity analysis showed a negligible (<1%) difference in the point estimates and CI of the results between the 2 analyses. A sensitivity analysis was also conducted with a 2-month wash-in period, which showed no appreciable difference in results (data not shown). Test of assumptions for use of linear regression showed no violations, data not shown. No autocorrelation was found across a 12-month period in any of the time series analyses.
Using the rate of platelet transfusion as a nonequivalent dependent control variable, we found that the rates of transfusion in the preintervention was not changing, with an estimate of 0.02 (95% CI, 0-0.03; P = .05) units per 100 pt day per month (Figure 3). As well, the rates of platelet transfusion in the postintervention period remained unchanged, with an estimate of −0.02 (95% CI, −0.05-0.01; P = .3). The difference between the rates in the pre- and postintervention periods was −0.03 (95% CI, −0.07-0.002; P = .06) units per 100 pt days per month, which was not statistically significant, and there was no change in the intercept following the intervention (P = .06) (Figure 3). Thus, we found no association with the intervention on the rate of platelet transfusion, or, taken another way, we found no change in the control group.

## Discussion

Multiple stakeholders, including physicians, health care systems, government, patient advocacy groups, and insurers are all promoting more thoughtful use of health care resources. Strong evidence from multiple randomized clinical trials has supported a more restrictive transfusion threshold for patients with asymptomatic anemia.
• Hebert P.C.
• Wells G.
• Blajchman M.A.
• et al.
A multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. Transfusion requirements in critical care investigators, Canadian Critical Care Trials Group.
• Carson J.L.
• Terrin M.L.
• Noveck H.
• et al.
Liberal or restrictive transfusion in high-risk patients after hip surgery.
• Villanueva C.
• Colomo A.
• Bosch A.
• et al.
Transfusion strategies for acute upper gastrointestinal bleeding.
• Holst L.B.
• Haase N.
• Wetterslev J.
• et al.
Lower versus higher hemoglobin threshold for transfusion in septic shock.
One method to improve adherence to guidelines has been the use of clinical decision support. In our study examining the implementation of a clinical decision support tool in a commercial EHR, we found an association with this implementation and a further reduction in the use of red blood cell transfusion in patients with hematocrit ≥21%. Of interest, we found that prior to implementation, a significant rate of decrease in red blood cell transfusion was already underway. This decrease was likely the result of other factors, possibly changing physician knowledge, and practice patterns likely being driven by strong efforts to reduce red blood cell transfusions use. In support of this notion, comparatively little effort has been undertaken to improve the use of platelets, and we found, in our use of platelets as a nonequivalent dependent control variable, no secular trend for change.
By examining the counterfactual, that is, the fictitious case in which the clinical decision support tool was not implemented, we can estimate the effects of the intervention. Evaluating the final month of the study, November 2014, had the clinical decision support tool not been implemented we may have expected a rate of 3.8 red blood cell units (w/ hematocrit ≥21%) transfused per 100 pt days, compared with 2.5 as actually observed. Given the average monthly census, this equates to approximately 138 red blood cell transfusions avoided and an estimated $100K-$200K in savings for the month of November 2014 alone.
Others have argued against the use of simple 2-group comparison statistics such as chi-squared or t tests in quality improvement assessments.
• Harris A.D.
• Baumgarten M.
• Zuckerman I.H.
• Fink J.C.
• Perencevich E.N.
The use and interpretation of quasi-experimental studies in infectious diseases.
• Harris A.D.
• McGregor J.C.
• Perencevich E.N.
• et al.
The use and interpretation of quasi-experimental studies in medical informatics.
• Penfold R.B.
• Zhang F.
Use of interrupted time series analysis in evaluating health care quality improvements.
• Wagner A.K.
• Soumerai S.B.
• Zhang F.
• Ross-Degnan D.
Segmented regression analysis of interrupted time series studies in medication use research.
• Fan E.
• Laupacis A.
• Pronovost P.J.
• Guyatt G.H.
• Needham D.M.
How to use an article about quality improvement.
In fact, other researchers have examined an intervention similar to ours, yet they utilized exactly those limited comparisons, with no accounting for secular trends and thus, no conclusions can be drawn from their results.
• Goodnough L.T.
• Shieh L.
• Cheng N.
• Khari P.
• Maggio P.
Improved blood utilization using real-time clinical decision support.
Using both chi-squared and t tests in our study, we find that implementation of the clinical decision support tool is associated with a reduction in use of red blood cells. However, further examination shows that the underlying rates for red blood cell transfusions both with the strict, hematocrit ≥21%, and more loose outcome measure, all hematocrit levels, there is already a significant secular trend underway prior to the intervention itself. Failure to account for this trend can easily result in incorrect association of the changed outcome with the intervention. One of the key strengths of interrupted time series analysis is that it accounts for and visually displays well the secular trends in the outcome measure and thus its use is particularly well suited to our purposes. After analyzing the results using the strict outcome measure, hematocrit ≥21%, we did indeed find a change in the magnitude of the decreasing rate associated with the intervention (Figure 3).
An additional strength of our study is the use of a nonequivalent dependent control variable. With interrupted time series analysis, changes in the attributes of the underlying population are accounted for as long as they do not occur contemporaneously with the intervention itself. When the latter occurs they are referred to as competing interventions. While use of a concurrent control group can account for this, many interventions, particularly quality improvement types, do not lend themselves to easily available local comparators. In these circumstances, use of a nonequivalent dependent control variable is one way to combat this problem.
• Cook T.
• Campbell D.
Experimental and Quasi-Experimental Designs.
• Harris A.D.
• Baumgarten M.
• Zuckerman I.H.
• Fink J.C.
• Perencevich E.N.
The use and interpretation of quasi-experimental studies in infectious diseases.
An ideal nonequivalent dependent control variable has the same construct, meaning the same causal and confounding influences, except the intervention itself. We utilized platelet transfusion as such. While visually examining Figure 3, there appears to be a difference in the slopes of the regression lines between the pre- and postintervention periods. However, the 95% confidence intervals for both regression lines contains zero. Therefore, the slopes of both lines are statistically no different than zero and no different from each other. While it is tempting to speculate about “nonsignificant” trends in the slopes, for the purpose of the analysis as intended, there is no difference. Furthermore, if the analysis is repeated with the outliers (platelet transfusion values >4 and <2) removed, the 95% confidence intervals of slopes for both regression lines narrow around zero (data not shown). Thus, given that we found no effect from the intervention on platelet transfusion, this further lessens any threats to internal validity.
We believe our study has several additional strengths. Firstly, it is a real-world implementation as it was implemented in a commercial EHR with no a priori plans for study and thus, most closely resembles what other institutions would replicate, that is, it has strong external validity. While randomized clinical trials are considered the gold standard for evaluating clinical interventions in many situations, they prove less than ideal. Their strict inclusion and exclusion criteria can make their results limited in terms of generalizability and they often have substantial financial and resource costs.
• Penfold R.B.
• Zhang F.
Use of interrupted time series analysis in evaluating health care quality improvements.
• Fan E.
• Laupacis A.
• Pronovost P.J.
• Guyatt G.H.
• Needham D.M.
How to use an article about quality improvement.
Additionally, with our increasing focus on understanding and improving health care delivery, developing robust yet practical methods for outcome analysis is becoming exceedingly important.
• Ash J.S.
• Sittig D.F.
• Dykstra R.
• Campbell E.
• Guappone K.
The unintended consequences of computerized provider order entry: findings from a mixed methods exploration.
Two key threats to internal validity in retrospective time series studies are instrumentation and selection. Instrumentation refers to the ability or method of making measurements, and selection concerns itself with external changes to the cohorts. Firstly, with instrumentation there were no significant changes in either the laboratory or blood banking systems or methods in which those data were recorded, and thus we do not think our ability to track transfusions or laboratory values was in any way compromised. Secondly, we feel confident that any changes in the groups are likely negligible, as seen from the characteristics in Table 1. One final limitation of this study is the reliance on secondary use of operational EHR data. Other researchers have outlined the potential limitations and pitfalls in use of this type of data.
• Hersh W.R.
• Weiner M.G.
• Embi P.J.
• et al.
Caveats for the use of operational electronic health record data in comparative effectiveness research.
Of note, when visually examining Figure 2, it appears that the second implementation (August 2013) of the clinical decision support tool to additional user groups (general surgery and bone marrow transplant) was associated with a larger effect on the reduction in red blood cell transfusion use than the initial implementation. It is conceivable that had the surgical and bone marrow service been historically higher users of inappropriate red blood cell transfusions, their inclusion resulted in the predominant effect seen in the study. Unfortunately, we are unable to perform a stratified analysis by clinical user group to determine the respective contribution from each implementation cohort, owing to the lack of this patient-level variable in the dataset.
In conclusion, we found the use of an evidence-based clinical decision support tool in a commercial EHR, Epic, reduced the overuse of red blood cell transfusions in our hospital. In this observational trial we used robust methods, specifically, interrupted time series analysis, to help strengthen our analysis, and utilized a nonequivalent dependent control variable to further reduce threats to internal validity. We believe this study represents meaningful work in that it is a real-world study and the intervention could be easily reproduced by other health care organizations on the same or even a different vendor system. Additionally, we believe our study serves as an example of the importance of using strong analytical methods for quasi-experimental studies.

## Acknowledgments

We thank Diana Pennington for her assistance in all aspects of the clinical decision support tool.

## References

• Pfuntner A.
• Wier L.
• Stocks C.
Most Frequent Procedures Performed in U.S. Hospitals, 2011.
Agency for Healthcare Quality and Research, Rockville, MD2013
• Department of Health and Human Services
The 2011 National Blood Collection and Utilization Survey Report.
Department of Health and Human Services, Washington, DC2013
• Shander A.
• Hofmann A.
• Ozawa S.
• Theusinger O.M.
• Gombotz H.
• Spahn D.R.
Activity-based costs of blood transfusions in surgical patients at four hospitals.
Transfusion. 2010; 50: 753-765
• Perkins H.A.
• Busch M.P.
Transfusion-associated infections: 50 years of relentless challenges and remarkable progress.
Transfusion. 2010; 50: 2080-2099
• Carson J.L.
• Grossman B.J.
• Kleinman S.
• et al.
Red blood cell transfusion: a clinical practice guideline from the AABB*.
Ann Intern Med. 2012; 157: 49-58
• Hebert P.C.
• Wells G.
• Blajchman M.A.
• et al.
A multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. Transfusion requirements in critical care investigators, Canadian Critical Care Trials Group.
N Engl J Med. 1999; 340: 409-417
• Hajjar L.A.
• Vincent J.L.
• Galas F.R.
• et al.
Transfusion requirements after cardiac surgery: the TRACS randomized controlled trial.
JAMA. 2010; 304: 1559-1567
• Carson J.L.
• Terrin M.L.
• Noveck H.
• et al.
Liberal or restrictive transfusion in high-risk patients after hip surgery.
N Engl J Med. 2011; 365: 2453-2462
• Villanueva C.
• Colomo A.
• Bosch A.
• et al.
Transfusion strategies for acute upper gastrointestinal bleeding.
N Engl J Med. 2013; 368: 11-21
• Holst L.B.
• Haase N.
• Wetterslev J.
• et al.
Lower versus higher hemoglobin threshold for transfusion in septic shock.
N Engl J Med. 2014; 371: 1381-1391
• Murphy G.J.
• Pike K.
• Rogers C.A.
• et al.
Liberal or restrictive transfusion after cardiac surgery.
N Engl J Med. 2015; 372: 997-1008
• Carson J.L.
• Sieber F.
• Cook D.R.
• et al.
Liberal versus restrictive blood transfusion strategy: 3-year survival and cause of death results from the FOCUS randomised controlled trial.
Lancet. 2015; 385: 1183-1189
• The Joint Commission
Proceedings from the National Summit on Overuse.
The Joint Commission, Oakbrook Terrace, IL2012
• Davis D.A.
• Thomson M.A.
• Oxman A.D.
• Haynes R.B.
Changing physician performance. A systematic review of the effect of continuing medical education strategies.
JAMA. 1995; 274: 700-705
• Roshanov P.S.
• Fernandes N.
• Wilczynski J.M.
• et al.
Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials.
BMJ. 2013; 346: f657
• Linden A.
Stata module for conducting interrupted time series analysis for single and multiple groups.
(Available at:) (Accessed July 23, 2015)
• Cook T.
• Campbell D.
Experimental and Quasi-Experimental Designs.
Houghton Mifflin, Boston2002
• Harris A.D.
• Baumgarten M.
• Zuckerman I.H.
• Fink J.C.
• Perencevich E.N.
The use and interpretation of quasi-experimental studies in infectious diseases.
Clin Infect Dis. 2004; 38: 1586-1591
• Harris A.D.
• McGregor J.C.
• Perencevich E.N.
• et al.
The use and interpretation of quasi-experimental studies in medical informatics.
J Am Med Inform Assoc. 2006; 13: 16-23
• Penfold R.B.
• Zhang F.
Use of interrupted time series analysis in evaluating health care quality improvements.
• Wagner A.K.
• Soumerai S.B.
• Zhang F.
• Ross-Degnan D.
Segmented regression analysis of interrupted time series studies in medication use research.
J Clin Pharm Ther. 2002; 27: 299-309
• Fan E.
• Laupacis A.
• Pronovost P.J.
• Guyatt G.H.
• Needham D.M.
How to use an article about quality improvement.
JAMA. 2010; 304: 2279-2287
• Goodnough L.T.
• Shieh L.
• Cheng N.
• Khari P.
• Maggio P.
Improved blood utilization using real-time clinical decision support.
Transfusion. 2014; 54: 1358-1365
• Ash J.S.
• Sittig D.F.
• Dykstra R.
• Campbell E.
• Guappone K.
The unintended consequences of computerized provider order entry: findings from a mixed methods exploration.
Int J Med Inform. 2009; 78: S69-S76
• Hersh W.R.
• Weiner M.G.
• Embi P.J.
• et al.
Caveats for the use of operational electronic health record data in comparative effectiveness research.
Med Care. 2013; 51: S30-S37