Advertisement

Understanding and Reducing Ceruloplasmin Overuse with a Decision Support Intervention for Liver Disease Evaluation

      Abstract

      Background

      Over-testing is a source of significant health care costs, both in terms of dollars spent and the false positives generated. Clinicians frequently test patients with elevated liver enzymes (a common problem) for Wilson disease (a rare disease) using a ceruloplasmin level.

      Methods

      We performed a prospective pre-post study between October 2013 and November 2014. We deployed a pop-up screen in our provider order entry system to present clinicians with the guidelines for and test characteristics of ceruloplasmin use. Outcomes included rate ratios for test utilization. Indications for inpatient orders were provided by ordering clinicians and reviewed.

      Results

      Ceruloplasmin was ordered 448 and 219 times in the pre- and postintervention periods, respectively. Rate ratios for orders from gastroenterologists and general internists were 0.49; 95% confidence interval (CI), 0.40-0.59, and 0.31; 95% CI, 0.20-0.50, respectively (P <.0001, for both). Following the intervention, there was an 82% and 40% decrease in orders by internists in the outpatient and inpatient settings, respectively. Tests of confirmation (necessary for positive ceruloplasmins) were more common in the group tested following the intervention, risk ratio 1.80; 95% CI, 1.04-3.08; P = .02. Of the residual ceruloplasmin orders by inpatient internists after the intervention, 75% (18/24) were recommended by hepatology consultants.

      Conclusion

      The optimal intervention for the reduction of over-testing should include automated changes to the ordering system combined with efforts to change testing culture through education.

      Keywords

      Clinical Significance
      • Over-testing is common, particularly for the evaluation of liver disease.
      • A pop-up screen in provider order entry simultaneously reduced overuse, improved the quality of utilization, and allowed for tracking testing rationale.
      • Wilson disease should be tested in patients who are younger than 55 years with liver disease of unknown etiology after a prior evaluation.
      Over-testing is a source of significant health care costs.
      • Smith M.
      • Saunders R.
      • Stuckhardt L.
      • McGinnis J.M.
      Best Care at Lower Cost: The Path to Continuously Learning Health Care in America.
      • Berwick D.M.
      • Hackbarth A.D.
      Eliminating waste in US health care.
      It is a form of medical overuse where health care is provided when the potential benefits are either negligible or outweighed by the risk of harm.
      • National Academy of Sciences
      Institute of Medicine, Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century.
      • Morgan D.J.
      • Wright S.M.
      • Dhruva S.
      Update on medical overuse.
      Nondirected testing is a specific form of over-testing, where a common complaint is assessed with a “panel” that includes a multitude of testable diseases.
      • Lichtenstein M.J.
      • Pincus T.
      How useful are combinations of blood tests in “rheumatic panels” in diagnosis of rheumatic diseases?.
      • Tapper E.B.
      • Rahni D.O.
      • Arnaout R.
      • Lai M.
      The overuse of serum ceruloplasmin measurement.
      • Tapper E.B.
      • Patwardhan V.R.
      • Curry M.
      Low yield and utilization of confirmatory testing in a cohort of patients with liver disease assessed for Alpha-1 antitrypsin deficiency.
      This pattern of testing is both expensive and associated with false positives, which, in turn, generate anxiety and emotional costs.
      • Lichtenstein M.J.
      • Pincus T.
      How useful are combinations of blood tests in “rheumatic panels” in diagnosis of rheumatic diseases?.
      • Tapper E.B.
      • Rahni D.O.
      • Arnaout R.
      • Lai M.
      The overuse of serum ceruloplasmin measurement.
      • Tapper E.B.
      • Patwardhan V.R.
      • Curry M.
      Low yield and utilization of confirmatory testing in a cohort of patients with liver disease assessed for Alpha-1 antitrypsin deficiency.
      • Welch H.G.
      • Passow H.J.
      Quantifying the benefits and harms of screening mammography.
      Nondirected testing is particularly common in the evaluation of elevated liver enzymes.
      • Tapper E.B.
      • Rahni D.O.
      • Arnaout R.
      • Lai M.
      The overuse of serum ceruloplasmin measurement.
      • Tapper E.B.
      • Patwardhan V.R.
      • Curry M.
      Low yield and utilization of confirmatory testing in a cohort of patients with liver disease assessed for Alpha-1 antitrypsin deficiency.
      The initial set of tests requested by clinicians frequently includes tests for viral and autoimmune hepatitis, iron overload, alpha-1 antitrypsin deficiency, and Wilson disease (WD).
      • Tapper E.B.
      • Rahni D.O.
      • Arnaout R.
      • Lai M.
      The overuse of serum ceruloplasmin measurement.
      • Tapper E.B.
      • Patwardhan V.R.
      • Curry M.
      Low yield and utilization of confirmatory testing in a cohort of patients with liver disease assessed for Alpha-1 antitrypsin deficiency.
      Guidelines suggest testing for WD with a ceruloplasmin level after excluding common liver diseases and, given the rarity of late-onset WD, rarely testing patients >55 years old.
      • Roberts E.A.
      • Schilsky M.L.
      Diagnosis and treatment of Wilson disease: an update.
      WD leads to chronic copper accumulation in the liver (among other sites) and is a rare cause of liver disease. While elevated liver enzymes are common (7.9% of Americans), WD is found in roughly 3 of 100,000 people (0.003%).
      • Tapper E.B.
      • Rahni D.O.
      • Arnaout R.
      • Lai M.
      The overuse of serum ceruloplasmin measurement.
      • Roberts E.A.
      • Schilsky M.L.
      Diagnosis and treatment of Wilson disease: an update.
      • Clark J.M.
      • Brancati F.L.
      • Diehl A.M.
      The prevalence and etiology of elevated aminotransferase levels in the United States.
      We previously assessed physician behavior regarding the testing of patients with elevated liver enzymes for WD.
      • Tapper E.B.
      • Rahni D.O.
      • Arnaout R.
      • Lai M.
      The overuse of serum ceruloplasmin measurement.
      We found that clinicians rarely follow guidelines, and as a result, ceruloplasmin's test performance suffers. In the 2 studies of large-scale ceruloplasmin testing for patients with liver disease evaluated at referral centers (encompassing 7890 patients), the combined yield was 9 patients with WD (0.1%).
      • Tapper E.B.
      • Rahni D.O.
      • Arnaout R.
      • Lai M.
      The overuse of serum ceruloplasmin measurement.
      • Cauza E.
      • Maier-Dobersberger T.
      • Polli C.
      • Kaserer K.
      • Kramer L.
      • Ferenci P.
      Screening for Wilson's disease in patients with liver diseases by serum ceruloplasmin.
      By testing patients with a low pretest probability, the proportions of positive results in these studies that were false positives were understandably high (94.1%-98.1%).
      • Tapper E.B.
      • Rahni D.O.
      • Arnaout R.
      • Lai M.
      The overuse of serum ceruloplasmin measurement.
      • Cauza E.
      • Maier-Dobersberger T.
      • Polli C.
      • Kaserer K.
      • Kramer L.
      • Ferenci P.
      Screening for Wilson's disease in patients with liver diseases by serum ceruloplasmin.
      We conducted a prospective pre-post study to evaluate the impact of a pop-up screen in provider order entry (POE) on ceruloplasmin utilization in the 7 months before and after its implementation between October 1, 2013 and November 27, 2014. Given that most tests are ordered using a computerized POE system, decision supports for POE interfaces have proven popular and are often effective.
      • Bates D.W.
      • Kuperman G.J.
      • Rittenberg E.
      • et al.
      A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests.
      • Bates D.W.
      • Kuperman G.J.
      • Wang S.
      • et al.
      Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality.
      • Hunt D.L.
      • Haynes R.B.
      • Hanna S.E.
      • Smith K.
      Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review.
      • Kaushal R.
      • Shojania K.G.
      • Bates D.W.
      Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review.
      • Lobach D.F.
      • Hammond W.
      Computerized decision support based on a clinical practice guideline improves compliance with care standards.
      Herein, we describe an intervention that allowed us to reduce the rate and understand the source of ceruloplasmin over-testing.

      Methods

      We prospectively reviewed the records of all patients for whom a serum ceruloplasmin was sent to the Beth Israel Deaconess Medical Center clinical laboratory between October 1, 2013 and November 27, 2014. Our clinical laboratory serves outpatient clinics, including a large primary care network; specialty clinics (eg, hepatology, neurology); and a 600-bed tertiary referral center in Boston. All “send-out” tests not performed in house are registered in the central laboratory clinical data repository. When a clinician chooses to order the test, the screen depicted in Figure 1 is presented, requesting clinicians to confirm their interest in the test. Clinicians had the option of ordering but were asked to confirm their interest in the test. For inpatient orders, the reason for the order must be provided to continue. This intervention was deployed on May 1, 2014. The general trends of ceruloplasmin utilization have been reviewed elsewhere.
      • Tapper E.B.
      • Sengupta N.
      • Lai M.
      • Horowitz G.
      A Decision Support Tool to Reduce Overtesting for Ceruloplasmin and Improve Adherence With Clinical Guidelines.
      This study was approved by the Beth Israel Deaconess Medical Center Institutional Review Board. The conduct of this study and preparation of this manuscript were performed in accordance with SQUIRE guidelines (Standards for QUality Improvement Reporting Excellence).
      • Davidoff F.
      • Batalden P.
      • Stevens D.
      • Ogrinc G.
      • Mooney S.
      Publication guidelines for quality improvement in health care: evolution of the SQUIRE project.
      Figure thumbnail gr1
      Figure 1The study intervention. When clinicians choose to order ceruloplasmin, they are faced with this screen as an interruption. BIDMC = Beth Israel Deaconess Medical Center; IRB = Institutional Review Board.

      Study Design

      We developed a prespecified stopping rule for data collection before analysis of outcomes based on a power calculation. We chose a 20% reduction in our primary outcome based on prior decision support studies.
      • van Wijk M.A.
      • van der Lei J.
      • Mosseveld M.
      • Bohnen A.M.
      • van Bemmel J.H.
      Assessment of decision support for blood test ordering in primary care: a randomized trial.
      • Bright T.J.
      • Wong A.
      • Dhurjati R.
      • et al.
      Effect of clinical decision-support systemsa systematic review.
      As hepatitis C screening became part of clinical practice since the development of this intervention, we powered our study based on simultaneous hepatitis B (HBV) testing (vis-à-vis viral load or serologic tests). In order to reduce the rate of simultaneous ceruloplasmin and HBV testing by 20%, using our previous experience as a guide,
      • Tapper E.B.
      • Rahni D.O.
      • Arnaout R.
      • Lai M.
      The overuse of serum ceruloplasmin measurement.
      our intervention would have to decrease the rate of simultaneous HBV and ceruloplasmin testing from 1.39 unique patients per day (4062 patients in 8 years) to 1.11 patients per day. To achieve 90% power to detect this difference with a significance level of 5%, it was estimated that 165 subjects before and after the intervention would be required. We achieved this number of subjects after 5 months, but had reviewed the data at ∼7 months (210 days), so included all tested subjects to that date. We then included a preintervention control period that was roughly 7 months (211 days) to serve as a control for pre-post analyses.

      Procedures

      For each ceruloplasmin order, we recorded the clinical and demographic details of the corresponding patients. These included age, sex, and alanine aminotransferase, aspartate aminotransferase, alkaline phosphatase, and total bilirubin levels. For each ceruloplasmin ordered, we assessed whether there was simultaneous testing (testing on the same blood draw) for any of the following: hepatitis B (surface antigen, surface antibody, core antibodies, viral load), hepatitis C (antibody or viral load), autoimmune hepatitis (antinuclear antibody, antismooth muscle antibody, immunoglobulin G), hemochromatosis (iron binding capacity, mutation screen), or alpha-1 antitrypsin deficiency (alpha-1 antitrypsin [AAT] levels and phenotype). Hepatitis B infection was defined as a positive surface antigen, viral load, or isolated positive test for immunoglobulin M antibody to hepatitis B core antigen. Hepatitis C infection was defined as a positive antibody or viral load; where available patients with a positive antibody and negative viral load were not considered to have hepatitis C. We also determined whether the patients were ever assessed for a urine copper concentration. All duplicate ceruloplasmin orders (more than one per patient) were disregarded.
      For each ceruloplasmin order, the ordering provider and indication were recorded. Each order was associated with a single billing International Classification of Diseases, 9th Revision code. International Classification of Diseases, 9th Revision codes were categorized as being liver related or neurological. Liver-related codes included 70, 79.99, 155, 235.4, 570, 571, 573, 576.1, 782.4, 787.99, 789, 790.4-.6, 793.3, 794.8, 796.4, 798.1, 995.3, E850.4, and E950. Neurologic codes included 2956, 314, 3313, 338.4, 340, 345, 348.8, 356, 368, 377.1, 719.46, 723.9, 729.5, 7801, and 786.9. Internists coded the remaining orders as routine examination (V70.0). Patients with known Wilson disease for whom ceruloplasmin levels were ordered were excluded (n = 4).
      All orders made on hospitalized patients in the intervention period were associated with a written indication by the ordering physician. A survey (Appendix, available online) was developed to assess the rationale behind ceruloplasmin ordering as the clinician's response to the intervention (Qualtrics, Provo, UT). The survey was sent to 20 randomly selected outpatient clinicians chosen from a list of clinicians who ordered ceruloplasmin in the pre- and postintervention periods.
      In accordance with American Association for the Study of Liver Disease Guidelines, we used 20 mg/dL as the lower end of the reference interval in our study and treated values <20 mg/dL as suggestive of WD.
      • Roberts E.A.
      • Schilsky M.L.
      Diagnosis and treatment of Wilson disease: an update.
      A diagnosis of WD was determined on the basis of the results of confirmatory testing (urine or hepatic copper levels, or both) and medical record documentation. We reviewed the charts of all patients who had a positive ceruloplasmin or any serum or urine copper measurement. Tests of confirmation included urine copper concentrations, which were considered positive when >40 mg/L in a 24-hour collection, and a hepatic copper determination for which a positive level was considered >50 mg/g dry weight in a liver biopsy. False-positive ceruloplasmin was defined as <20 mg/dL with a negative urine copper or hepatic copper measurement or a documented alternate diagnosis by a hepatologist.
      • Roberts E.A.
      • Schilsky M.L.
      Diagnosis and treatment of Wilson disease: an update.
      Finally, we also determined the number of AAT tests ordered during the study period for patients older than 55 years of age. AAT is often ordered to evaluate liver disease of unknown etiology, usually in a nondirected fashion.
      • Tapper E.B.
      • Patwardhan V.R.
      • Curry M.
      Low yield and utilization of confirmatory testing in a cohort of patients with liver disease assessed for Alpha-1 antitrypsin deficiency.
      As such, we used AAT ordering patterns as a marker of the secular trend in liver enzyme evaluation against which we could compare trends in ceruloplasmin use.

      Analysis

      All data were analyzed using JMP PRO SAS 11 (SAS Institute Inc, Cary, NC). Descriptive statistics included mean and standard deviation for variables with normal distributions, and median with interquartile range for nonparametric variables. Normal variables were compared using the Student's t test, while the Wilcoxon rank-sums test was employed for nonparametric variables. Proportions were tested with Fisher's exact test and relative risk ratios where the information was pertinent. Rates were compared with rate ratios with time denominators specific to the length of observation in the pre- (211 days) and postintervention periods (210 days). Graphical representations of the data were performed using GraphPad Prism, version 6 (La Jolla, CA).

      Results

      Patient characteristics are detailed in Table 1. Ceruloplasmin was ordered 448 and 219 times in the pre- and postintervention periods, respectively. The rate for liver-related indications for ceruloplasmin orders decreased following the intervention (Figure 2). Similarly, orders from both gastroenterologists and internists decreased substantially (Figure 2). Overall, the rate ratios for orders from gastroenterologists and general internists was 0.49; 95% CI, 0.40-0.59, and 0.31; 95% CI, 0.20-0.50, respectively (P value for both <.0001). Before the intervention, internists ordered 34 tests in the outpatient setting, compared with 6 in the postintervention period, rate ratio 0.18; 95% CI, 0.07-0.40, P <.0001. The rate of ceruloplasmin's involvement in nondirected testing decreased after the intervention. Simultaneous orders for other liver tests were significantly lower (Figure 3). Additionally, compared with a control test (Alpha-1 antitrypsin), the rate of ceruloplasmin testing in patients older than 55 years decreased substantially (Figure 3).
      Table 1Basic Clinical Characteristics of the Patients Tested Before and After the Study Intervention
      PreinterventionPostinterventionP-Value
      Number tested448219<.0001
      Age (y ± SD)47.6 ± 14.644.4 ± 15.2.01
      Alanine aminotransferase (IU/L) – median (IQR)58 (36-104)59 (35-116).9
      Aspartate aminotransferase (IU/L) – median (IQR)46 (31-85)47 (33-83)1
      Alkaline phosphatase (IU/L) – median (IQR)95 (67-131)89 (68-131).7
      Bilirubin (mg/dL) – median (IQR)0.6 (0.4-1.2)0.7 (0.5-1.2).3
      INR – median (IQR)1.1 (1-1.4)1.1 (1-1.5).7
      Physician's department/division
       Gastroenterology334 (74.6%)162 (74.0%).9
       Neurology25 (5.6%)20 (9.1%).05
       General internists74 (16.5%)23 (10.5%).02
      IQR = interquartile range; INR = international normalized ratio; IU = international unit; L = liter; SD = standard deviation.
      Figure thumbnail gr2
      Figure 2Ceruloplasmin orders by diagnosis and ordering provider. The clinician's response to the intervention is depicted according to the number of tests ordered for liver disease diagnoses (left) and by the clinician's division (right). These data show that the number of tests ordered for liver-specific indications, and by both internists and gastroenterologists, decreased substantially. The arrow indicates the start of the intervention.
      Figure thumbnail gr3
      Figure 3The intervention achieved a significant improvement in guideline adherence. Left: The rate of testing for patients older than 55 years decreased significantly following the intervention. This decrease was specific to ceruloplasmin as the rate of alpha-1 antitrypsin orders (AAT) did not change. Right: The rate of simultaneous testing for other liver diseases decreased significantly after the intervention. The arrow indicates the start of the intervention.
      The results of ceruloplasmin testing are detailed in Table 2. The proportions of positive ceruloplasmins were 7.4% (n = 33) and 10.5% (n = 23) in the pre- and postintervention periods, respectively (P = .09). Tests of confirmation were more common in the group tested following the intervention; risk ratio 1.80; 95% CI, 1.04-3.08, P = .02. All positive ceruloplasmins (regardless of study phase) were false positives; no new diagnoses of WD were made during the study period. The rate of false-positive tests decreased from 0.16 to 0.11 per day, rate ratio 0.70; 95% CI, 0.41-1.10, P = .19. Ultimate diagnoses were obtained in all but 1 patient lost to follow-up (Table 2). Among the 28 unconfirmed positive tests, the clinical diagnoses included drug-induced liver injury and alcoholic cirrhosis (5 each), hepatitis B (4), acetaminophen-induced acute liver injury (3), pancreatic cancer with cholestatic liver injury, hemochromatosis, and Gilbert's disease without hepatitis (1 each). Eight patients received a diagnosis of nonalcoholic steatohepatitis, 5 of whom were aged 55 years or less. These patients had ceruloplasmin levels of 19 (n = 4) and 18 (n = 1). The following proportions of patients pre- and postintervention received diagnoses of hepatitis B (7 [1.6%] vs 4 [1.8%]), hepatitis C (13 [2.9%] vs 7 [5.9%]), and hemochromatosis (41 [9.2%] vs 24 [11.0%]).
      Table 2Rates and Result of Investigation for Patients with Positive Ceruloplasmin Tests
      PreinterventionPostintervention
      Number tested448219
      Positive ceruloplasmin (<20 mg/dL) – n (%)33 (7.4%)23 (10.5%)
      Test of confirmation – n (%)12 (36.4%)15 (65.2%)
      Ultimate diagnoses9 – NASH6 – NASH
      8 – Alcohol5 – Alcohol
      5 – Idiosyncratic DILI4 – Hepatitis B
      3 – Tylenol2 – Tylenol
      2 – Hepatitis B1 – Alpha-1 antitrypsin deficiency, hemochromatosis, primary sclerosing cholangitis, Epstein-Barr virus, muscle, unknown (lost to follow-up)
      1 – Hepatitis A, hemochromatosis, glycogenosis, Gilbert's, muscle, pancreatic cancer
      DILI = drug-induced liver injury; NASH = nonalcoholic steatohepatitis.
      In the postintervention period, 57 patients had ceruloplasmin levels ordered as inpatients, comprising 26.0% of the postintervention orders. The indications provided for these tests were reviewed. Twenty-seven (35.1%) were ordered by physicians on a dedicated liver service, 17 of which were ordered as part of a transplant evaluation (required by a local protocol). Twenty-four (42.1%) were ordered by internists for patients with acute liver injury, 18 of which were ordered at the request of a consulting hepatologist. Only 6 (10.5%) tests were for patients with neuropsychiatric syndromes. Fifty-six (98.2%) ceruloplasmins were ordered in a nondirected fashion. Before the intervention, internists ordered 40 ceruloplasmins for inpatients, vs 24 following the intervention, rate ratio 0.60; 95% CI, 0.36-0.99, P = .04.
      A survey was administered to 20 randomly selected physicians who ordered ceruloplasmins in the outpatient setting to determine their testing rationale and response to the intervention (Appendix, available online). Fifteen physicians (75%) responded: 8 internists (5 staff, 3 trainees) and 7 gastroenterologists (3 staff, 4 trainees). Eleven (73.3%) found the intervention helpful. Seven (46.7%) clinicians (including 4 staff; 2 each in gastroenterology and internal medicine) felt that the intervention changed their ceruloplasmin ordering practice; 6 of whom felt that it changed the way they evaluate liver disease in general. Reasons for ceruloplasmin ordering in older patients included 3 clinicians citing seeing referral patients with prior negative evaluation and 1 clinician who felt that WD could present atypically. Only 20% (n = 3) reported planning to use ceruloplasmin in the first pass of liver disease evaluation.

      Discussion

      In this prospective trial, a novel decision support tool improved test utilization by reducing over-testing. We found that a simple (inexpensive) intervention that interrupts workflow can effect large changes in testing behavior. Furthermore, we present a design that allows for the tracking of testing rationale that will inform future interventions.
      These data confirm and extend the findings of prior work on decision support in 3 principle ways. First, we demonstrate how an intervention can improve utilization while determining the rationale for testing behavior. This is seen in a few ways. First, we reduced overuse. Second, we saw a general improvement in testing behavior with increased utilization of confirmatory tests for positive ceruloplasmins. Third, the tracking function allows us to both contrast and explain the difference in reduction of utilization by outpatient internists (82% reduction) and inpatient internists (40% reduction). We learned that residual testing by inpatient internists is driven by specialist recommendations. This experience suggests that the impact of automated interventions is limited by the testing culture and opinion of thought leaders. Automated interventions must therefore be combined with “soft” approaches such as educational conferences and recommendations from divisional leadership.
      Second, our success underscores the crucial contribution from intervention design. While well-designed decision support interventions reproducibly reduce health care overutilization, some are more effective than others.
      • O'Connor A.M.
      • Bennett C.L.
      • Stacey D.
      • et al.
      Decision aids for people facing health treatment or screening decisions.
      Like ours, the ideal intervention is a simple, clear, force-function that is positioned perfectly within the physician's workflow, provides clear and direct information about the rationale and impact of an intervention, and is associated with a testable outcome.
      • Bates D.W.
      • Kuperman G.J.
      • Wang S.
      • et al.
      Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality.
      • O'Connor A.M.
      • Bennett C.L.
      • Stacey D.
      • et al.
      Decision aids for people facing health treatment or screening decisions.
      Third, our study extends the role of decision support into a specific and common form of overuse where common clinical scenarios are addressed with nondirected testing. Our findings could inform programs to reduce nondirected testing where such behavior is commonplace, including the evaluation of rheumatologic disease (eg, rheumatoid factor,
      • Shmerling R.H.
      • Delbanco T.L.
      How useful is the rheumatoid factor?: an analysis of sensitivity, specificity, and predictive value.
      antinuclear antibody subtypes,
      • Solomon D.H.
      • Kavanaugh A.J.
      • Schur P.H.
      Evidence-based guidelines for the use of immunologic tests: antinuclear antibody testing.
      antineutrophil cytoplasmic antibodies
      • Rao J.K.
      • Weinberger M.
      • Oddone E.Z.
      • Allen N.B.
      • Landsman P.
      • Feussner J.R.
      The role of antineutrophil cytoplasmic antibody (c-ANCA) testing in the diagnosis of Wegener granulomatosis: a literature review and meta-analysis.
      • Mandl L.A.
      • Solomon D.H.
      • Smith E.L.
      • Lew R.A.
      • Katz J.N.
      • Shmerling R.H.
      Using antineutrophil cytoplasmic antibody testing to diagnose vasculitis: can test-ordering guidelines improve diagnostic accuracy?.
      ), serum heavy metal tests,
      • Schober S.E.
      • Sinks T.H.
      • Jones R.L.
      • et al.
      Blood mercury levels in US children and women of childbearing age, 1999-2000.
      renal ultrasounds for kidney injury,
      • Licurse A.
      • Kim M.C.
      • Dziura J.
      • et al.
      Renal ultrasonography in the evaluation of acute kidney injury: developing a risk stratification framework.
      and routine blood tests on stable inpatients.
      • May T.A.
      • Clancy M.
      • Critchfield J.
      • et al.
      Reducing unnecessary inpatient laboratory testing in a teaching hospital.
      • Stuebing E.A.
      • Miner T.J.
      Surgical vampires and rising health care expenditure: reducing the cost of daily phlebotomy.
      Our findings must be understood in the context of the study design. First, our results may not be generalizable to other centers with variable approaches to liver disease. Second, this intervention is limited to tests for which clear clinical performance data are available for presentation to the ordering clinician. Furthermore, given the proportion of tests ordered as part of a liver transplant evaluation protocol and for acute liver injury in ill inpatients, this invention is neither unlikely to nor should it necessarily change behavior where perceived urgency stimulates nondirected testing. Further study is needed to determine the optimal placement of decision support for conditions susceptible to nondirected testing, including targeted educational initiatives, reflex testing of rare conditions after common diseases have been excluded, and restricted ordering frameworks that avail clinicians of relatively few options.
      • van Wijk M.A.
      • van der Lei J.
      • Mosseveld M.
      • Bohnen A.M.
      • van Bemmel J.H.
      Assessment of decision support for blood test ordering in primary care: a randomized trial.
      We cannot provide information on the number of clinicians who were confronted with the decision support tool but did not order the test. Third, in this single-center study, although the intervention affected hundreds of clinicians, we cannot comment on the generalizability of the observed approach to abnormal liver enzymes. Finally, these data cannot speak to the concern that interventions to reduce ceruloplasmin testing may result in missed WD diagnoses. It should be noted, however, that over-testing does not, itself, capture all WD; ceruloplasmin is imperfectly sensitive,
      • Cauza E.
      • Maier-Dobersberger T.
      • Polli C.
      • Kaserer K.
      • Kramer L.
      • Ferenci P.
      Screening for Wilson's disease in patients with liver diseases by serum ceruloplasmin.
      • Ala A.
      • Borjigin J.
      • Rochwarger A.
      • Schilsky M.
      Wilson disease in septuagenarian siblings: raising the bar for diagnosis.
      and positive results are often treated by clinicians as false positives.
      • Tapper E.B.
      • Rahni D.O.
      • Arnaout R.
      • Lai M.
      The overuse of serum ceruloplasmin measurement.
      We firmly believe that WD should be considered in patients with undifferentiated liver disease, particularly those <55 years old, including, for example, patients with suspected nonalcoholic fatty liver disease who fail to improve their liver tests with weight loss.
      In summary, a simple decision support tool reduced over-testing for Wilson disease. A pop-up screen intervention can be used in other conditions where nondirected testing is common.

      Acknowledgment

      EBT had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. The authors thank Dr. J. Thomas Lamont for his guidance and mentorship.

      Appendix

      References

        • Smith M.
        • Saunders R.
        • Stuckhardt L.
        • McGinnis J.M.
        Best Care at Lower Cost: The Path to Continuously Learning Health Care in America.
        National Academies Press, Washington, DC2013
        • Berwick D.M.
        • Hackbarth A.D.
        Eliminating waste in US health care.
        JAMA. 2012; 307: 1513-1516
        • National Academy of Sciences
        Institute of Medicine, Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century.
        National Academies Press, Washington, DC2001
        • Morgan D.J.
        • Wright S.M.
        • Dhruva S.
        Update on medical overuse.
        JAMA Intern Med. 2015; 175: 120-124
        • Lichtenstein M.J.
        • Pincus T.
        How useful are combinations of blood tests in “rheumatic panels” in diagnosis of rheumatic diseases?.
        J Gen Intern Med. 1988; 3: 435-442
        • Tapper E.B.
        • Rahni D.O.
        • Arnaout R.
        • Lai M.
        The overuse of serum ceruloplasmin measurement.
        Am J Med. 2013; 126: 926.e1-926.e5
        • Tapper E.B.
        • Patwardhan V.R.
        • Curry M.
        Low yield and utilization of confirmatory testing in a cohort of patients with liver disease assessed for Alpha-1 antitrypsin deficiency.
        Dig Dis Sci. 2015; 60: 1589-1594
        • Welch H.G.
        • Passow H.J.
        Quantifying the benefits and harms of screening mammography.
        JAMA Intern Med. 2014; 174: 448-454
        • Roberts E.A.
        • Schilsky M.L.
        Diagnosis and treatment of Wilson disease: an update.
        Hepatology. 2008; 47: 2089-2111
        • Clark J.M.
        • Brancati F.L.
        • Diehl A.M.
        The prevalence and etiology of elevated aminotransferase levels in the United States.
        Am J Gastroenterol. 2003; 98: 960-967
        • Cauza E.
        • Maier-Dobersberger T.
        • Polli C.
        • Kaserer K.
        • Kramer L.
        • Ferenci P.
        Screening for Wilson's disease in patients with liver diseases by serum ceruloplasmin.
        J Hepatol. 1997; 27: 358-362
        • Bates D.W.
        • Kuperman G.J.
        • Rittenberg E.
        • et al.
        A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests.
        Am J Med. 1999; 106: 144-150
        • Bates D.W.
        • Kuperman G.J.
        • Wang S.
        • et al.
        Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality.
        J Am Med Inform Assoc. 2003; 10: 523-530
        • Hunt D.L.
        • Haynes R.B.
        • Hanna S.E.
        • Smith K.
        Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review.
        JAMA. 1998; 280: 1339-1346
        • Kaushal R.
        • Shojania K.G.
        • Bates D.W.
        Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review.
        Arch Intern Med. 2003; 163: 1409-1416
        • Lobach D.F.
        • Hammond W.
        Computerized decision support based on a clinical practice guideline improves compliance with care standards.
        Am J Med. 1997; 102: 89-98
        • Tapper E.B.
        • Sengupta N.
        • Lai M.
        • Horowitz G.
        A Decision Support Tool to Reduce Overtesting for Ceruloplasmin and Improve Adherence With Clinical Guidelines.
        JAMA Intern Med. 2015;
        • Davidoff F.
        • Batalden P.
        • Stevens D.
        • Ogrinc G.
        • Mooney S.
        Publication guidelines for quality improvement in health care: evolution of the SQUIRE project.
        Qual Saf Health Care. 2008; 17: i3-i9
        • van Wijk M.A.
        • van der Lei J.
        • Mosseveld M.
        • Bohnen A.M.
        • van Bemmel J.H.
        Assessment of decision support for blood test ordering in primary care: a randomized trial.
        Ann Intern Med. 2001; 134: 274-281
        • Bright T.J.
        • Wong A.
        • Dhurjati R.
        • et al.
        Effect of clinical decision-support systemsa systematic review.
        Ann Intern Med. 2012; 157: 29-43
        • O'Connor A.M.
        • Bennett C.L.
        • Stacey D.
        • et al.
        Decision aids for people facing health treatment or screening decisions.
        Cochrane Database Syst Rev. 2009; : CD001431
        • Shmerling R.H.
        • Delbanco T.L.
        How useful is the rheumatoid factor?: an analysis of sensitivity, specificity, and predictive value.
        Arch Intern Med. 1992; 152: 2417-2420
        • Solomon D.H.
        • Kavanaugh A.J.
        • Schur P.H.
        Evidence-based guidelines for the use of immunologic tests: antinuclear antibody testing.
        Arthritis Care Res. 2002; 47: 434-444
        • Rao J.K.
        • Weinberger M.
        • Oddone E.Z.
        • Allen N.B.
        • Landsman P.
        • Feussner J.R.
        The role of antineutrophil cytoplasmic antibody (c-ANCA) testing in the diagnosis of Wegener granulomatosis: a literature review and meta-analysis.
        Ann Intern Med. 1995; 123: 925-932
        • Mandl L.A.
        • Solomon D.H.
        • Smith E.L.
        • Lew R.A.
        • Katz J.N.
        • Shmerling R.H.
        Using antineutrophil cytoplasmic antibody testing to diagnose vasculitis: can test-ordering guidelines improve diagnostic accuracy?.
        Arch Intern Med. 2002; 162: 1509-1514
        • Schober S.E.
        • Sinks T.H.
        • Jones R.L.
        • et al.
        Blood mercury levels in US children and women of childbearing age, 1999-2000.
        JAMA. 2003; 289: 1667-1674
        • Licurse A.
        • Kim M.C.
        • Dziura J.
        • et al.
        Renal ultrasonography in the evaluation of acute kidney injury: developing a risk stratification framework.
        Arch Intern Med. 2010; 170: 1900-1907
        • May T.A.
        • Clancy M.
        • Critchfield J.
        • et al.
        Reducing unnecessary inpatient laboratory testing in a teaching hospital.
        Am J Clin Pathol. 2006; 126: 200-206
        • Stuebing E.A.
        • Miner T.J.
        Surgical vampires and rising health care expenditure: reducing the cost of daily phlebotomy.
        Arch Surg. 2011; 146: 524-527
        • Ala A.
        • Borjigin J.
        • Rochwarger A.
        • Schilsky M.
        Wilson disease in septuagenarian siblings: raising the bar for diagnosis.
        Hepatology. 2005; 41: 668-670