Effects of adjustment for referral bias on the sensitivity and specificity of single photon emission computed tomography for the diagnosis of coronary artery disease



      Referral bias, in which the result of a diagnostic test affects the subsequent referral for a more definitive test, influences the accuracy of noninvasive tests for coronary artery disease. This study evaluates the effect of referral bias on the apparent accuracy of single photon emission computed tomography (SPECT).

      Subjects and methods

      Over a 10-year period, 14 273 patients without known coronary artery disease underwent stress SPECT. Coronary angiography was performed within 3 months after the stress test in 1853 patients (13%). The apparent sensitivity, specificity, and likelihood ratios of SPECT were determined in these patients, and then adjusted for referral bias using two different formulas.


      The overwhelming majority (95%) of patients who underwent angiography had abnormal SPECT images. Apparent values for test indices were a sensitivity of 98%, a specificity of 13%, a likelihood ratio for a positive test of 1.1, and a likelihood ratio for a negative test of 0.15. Test indices adjusted for referral bias (using the two methods) were a sensitivity of 65% or 67%, a specificity of 67% or 75%, a likelihood ratio for a positive test of 2.0 or 2.7, and a likelihood ratio for a negative test of 0.44 or 0.52.


      Referral bias has a marked effect on the apparent accuracy of stress SPECT for the diagnosis of coronary disease. Adjustment for referral bias yields estimates for sensitivity and specifity and likelihood ratios that better reflect the accuracy of the technique.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to The American Journal of Medicine
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Ransohoff D.F
        • Feinstein A.R
        Problems of spectrum and bias in evaluating the efficacy of diagnostic tests.
        N Engl J Med. 1978; 299: 926-930
        • Philbrick J.T
        • Horwitz R.I
        • Feinstein A.R
        Methodologic problems of exercise testing for coronary artery disease.
        Am J Cardiol. 1980; 46: 807-812
        • Philbrick J.T
        • Horwitz R.I
        • Feinstein A.R
        • et al.
        The limited spectrum of patients’ studies in exercise test research.
        JAMA. 1982; 248: 2467-2470
        • Rozanski A
        • Diamond G.A
        • Berman D
        • et al.
        The declining specificity of exercise radionuclide ventriculography.
        N Engl J Med. 1983; 309: 518-522
        • Begg C.B
        • Greenes R.A
        Assessment of diagnostic tests when disease verification is subject to selection bias.
        Biometrics. 1983; 39: 207-215
        • Sox Jr, H.C
        Probability theory in the use of diagnostic tests. An introduction to critical study of the literature.
        Ann Intern Med. 1986; 104: 60-66
        • Diamond G.A
        Reverend Bayes’ silent majority.
        Am J Cardiol. 1986; 5: 1175-1180
        • Tavel M.E
        • Enas N.H
        • Wood J.R
        Sensitivity and specificity of tests.
        Am J Cardiol. 1987; 60: 1167-1169
        • Detrano R
        • Janos A
        • Lyons K.P
        • et al.
        Factors affecting sensitivity and specificity of a diagnostic test.
        Am J Med. 1988; 84: 699-710
        • Diamond G.A
        How accurate is SPECT thallium scintigraphy?.
        J Am Coll Cardiol. 1990; 16: 1017-1021
        • Diamond G.A
        Affirmative actions.
        Med Decis Making. 1991; 11: 48-56
        • Diamond G.A
        Off Bayes.
        Med Decis Making. 1992; 12: 22-31
        • The Institute of Medicine
        Assessment of Diagnostic Technology in Health Care. Rationale, Methods, Problems, and Directions. Council of Health Care Technology, Washington, DC1989
        • Gibbons R.J
        • Balady G.J
        • Beasley J.W
        • et al.
        ACC/AHA guidelines for exercise testing.
        J Am Coll Cardiol. 1997; 30: 260-315
        • Gibbons R.J
        • Chatterjee K
        • Daley J
        • et al.
        ACC/AHA/ACP-ASIM guidelines for the management of patients with chronic stable angina.
        J Am Coll Cardiol. 1999; 33: 2092-2197
        • Douglas P.S
        Is noninvasive testing for coronary artery disease accurate?.
        Circulation. 1997; 95: 299-302
        • Froelicher V.F
        • Fearon W.F
        • Ferguson C.M
        • et al.
        Lessons learned from studies of the standard exercise ECG test.
        Chest. 1999; 116: 1442-1451
        • Schwartz R.S
        • Jackson W.G
        • Celio P.V
        • et al.
        Accuracy of exercise 201-Tl myocardial scintigraphy in asymptomatic young men.
        Circulation. 1993; 87: 165-172
        • Morise A.P
        • Diamond G.A
        Comparison of the sensitivity and specificity of exercise electrocardiography in biased and unbiased populations of men and women.
        Am Heart J. 1995; 130: 741-747
        • Cecil M.P
        • Kosinski A.S
        • Jones M.T
        • et al.
        The importance of work-up (verification) bias correction in assessing the accuracy of SPECT thallium-201 testing for the diagnosis of coronary artery disease.
        J Clin Epidemiol. 1996; 49: 735-742
        • Roger V.L
        • Pellikka P.A
        • Bell M.R
        • et al.
        Sex and test verification bias.
        Circulation. 1997; 95: 405-410
        • Santana-Boado C
        • Candell-Riera J
        • Castell-Conesa J
        • et al.
        Diagnostic accuracy of technetium-99m-MIBI myocardial SPECT in women and men.
        J Nucl Med. 1998; 39: 751-755
        • Hirzel H.O
        • Senn M
        • Nuesch K
        • et al.
        Thallium-201 scintigraphy in complete left bundle branch block.
        Am J Cardiol. 1984; 53: 764-769
        • Lakkis N.M
        • He Z-X
        • Verani M.S
        Diagnosis of coronary artery disease by exercise thallium-201 tomography in patients with a right ventricular pacemaker.
        J Am Coll Cardiol. 1997; 29: 1221-1225
        • Bailey I.K
        • Come P.C
        • Kelly D.T
        • et al.
        Thallium-201 myocardial perfusion imaging in aortic valve stenosis.
        Am J Cardiol. 1977; 40: 889-899
        • Christian T.F
        • Miller T.D
        • Bailey K.R
        • Gibbons R.J
        Noninvasive identification of severe coronary artery disease using exercise tomographic thallium 201 imaging.
        Am J Cardiol. 1992; 70: 14-20
        • Cannan C.R
        • Miller T.D
        • Christian T.F
        • et al.
        Prognosis with abnormal thallium images in the absence of significant coronary artery disease.
        Am J Cardiol. 1992; 70: 1276-1280
        • Johnston D.L
        • Daley J.R
        • Hodge D.O
        • et al.
        Hemodynamic responses and adverse effects associated with adenosine and dipyridamole pharmacologic stress testing.
        Mayo Clin Proc. 1995; 70: 331-336
        • Principal Investigators of CASS and Their Associates
        The National Heart, Lung, and Blood Institute Coronary Artery Surgery Study (CASS).
        Circulation. 1981; 63: I1-I81
        • White C.W
        • Wright C.B
        • Doty D.B
        • et al.
        Does visual interpretation of the coronary arteriogram predict the physiologic importance of a coronary stenosis?.
        N Engl J Med. 1984; 310: 819-824
        • Brown K.A
        Prognostic value of thallium-201 myocardial perfusion imaging.
        Circulation. 1991; 83: 363-381
        • Iskandrian A.S
        • Chae S.C
        • Heo J
        • et al.
        Independent and incremental prognostic value of exercise single-photon emission computed tomographic (SPECT) thallium imaging in coronary artery disease.
        J Am Coll Cardiol. 1993; 22: 665-670
        • Hachamovitch R
        • Berman D.S
        • Shaw L.J
        • et al.
        Incremental prognostic value of myocardial perfusion SPECT for the prediction of cardiac death.
        Circulation. 1998; 97 ([published correction appears in Circulation 1998;98:120]): 533-543
        • Iskander S
        • Iskandrian A.E
        Risk assessment using single-photon emission computed tomographic technetium-99m sestamibi imaging.
        J Am Coll Cardiol. 1998; 32: 57-62
        • Okada R.D
        • Boucher C.A
        • Strauss H.W
        • Pohost G.M
        Exercise radionuclide imaging approaches to coronary artery disease.
        Am J Cardiol. 1980; 46: 1188-1204
        • Iskandrian A.S
        • Heo J
        • Kong B
        • Lyons E
        Effect of exercise level on the ability of thallium-201 tomographic imaging in detecting coronary artery disease.
        J Am Coll Cardiol. 1989; 14: 1477-1486
        • Gould K.L
        How accurate is thallium exercise testing for the diagnosis of coronary artery disease?.
        J Am Coll Cardiol. 1989; 14: 1487-1489
        • Fleischmann K.E
        • Hunink M.G.M
        • Kuntz K.M
        • Douglas P.S
        Exercise echocardiography or exercise SPECT imaging? A meta-analysis of diagnostic test performance.
        JAMA. 1998; 280: 913-920
        • Vogel R.A
        • Kirch D.L
        • LeFree M.T
        • et al.
        Thallium-201 myocardial perfusion scintigraphy.
        Am J Cardiol. 1979; 43: 787-793
        • Rizi H.R
        • Kline R.C
        • Thrall J.H
        • et al.
        Thallium-201 myocardial scintigraphy.
        J Nucl Med. 1981; 22: 493-499
        • Massie B.M
        • Wisneski J
        • Hollenberg M
        • et al.
        Quantitative analysis of seven-pinhole tomographic thallium-201 scintigrams.
        J Am Coll Cardiol. 1984; 3: 1178-1186
        • Tamaki N
        • Yonekura Y
        • Mukai T
        • et al.
        Segmental analysis of stress thallium myocardial emission tomography for localization of coronary artery disease.
        Eur J Nucl Med. 1984; 9: 99-105
        • Starling M.R
        • Dehmer G.J
        • Lancaster J.L
        • et al.
        Segmental coronary artery disease.
        Radiology. 1985; 157: 231-237
        • Uhl G.S
        • Kay T.D
        • Hickman J.R
        Computer-enhanced thallium scintigrams in asymptomatic men with abnormal exercise test.
        Am J Cardiol. 1981; 48: 1037-1043
        • DePasquale E.E
        • Nody A.C
        • DePuey E.G
        • et al.
        Quantitative rotational thallium-201 tomography for identifying and localizing coronary artery disease.
        Circulation. 1988; 77: 316-327
        • Froelicher V.F
        • Lehmann K.G
        • Thomas R
        • et al.
        The electrocardiographic exercise test in a population with reduced workup bias.
        Ann Intern Med. 1998; 128: 965-974
        • Lewis J.F
        • Lin L
        • McGorray S
        • et al.
        Dobutamine stress echocardiography in women with chest pain. Pilot phase data from the National Heart, Lung and Blood Institute Women’s Ischemia Syndrome Evaluation (WISE).
        J Am Coll Cardiol. 1999; 33: 1462-1468