Advertisement

Diagnostic Accuracy in the 21st Century - No Time for Conceit

  • Ami Schattner
    Correspondence
    Requests for reprints should be addressed to Ami Schattner, MD, Professor of Medicine, Hebrew University-Hadassah Medical School, Kaplan Medical Center, 76100 Rehovot, Jerusalem, 91120, Israel.
    Affiliations
    The Faculty of Medicine, Hebrew University and Hadassah Medical School, Jerusalem, Israel
    Search for articles by this author
      Physicians tend to think well of themselves and overrate their own performance. When asked about their handwashing rates before patient contact, the estimate of 73% (range 50%-95%) was nowhere near the observed frequency of just 9%.
      Handwashing Liaison Group
      Hand washing. A modest measure – with big effects.
      Asked to judge the proportion of patients under their care who had been screened for colorectal cancer according to guidelines, primary physicians’ estimate (>25%) contrasted sharply with reality (1.7%).
      • Schattner A
      • Gilad A.
      Primary care physicians’ awareness and implementation of screening guidelines for colorectal cancer.
      Moreover, most physicians interviewed strongly endorse shared decision-making, but in clinical practice it is often neglected in many settings.
      • Zeuner R
      • Frosch DL
      • Kuzemchak MD
      • Politi MC.
      Physicians’ perceptions of shared decision-making behaviours: a qualitative study demonstrating the continued chasm between aspirations and clinical practice.
      Physicians in intensive care units confidently reported daily pain and delirium assessment of most patients (70% and 53%, respectively), when in fact, only 20% and <1%, respectively of patients had such reports documented.
      • Chen K
      • Yang YL
      • Li HL
      • et al.
      A gap existed between physicians’ perceptions and performance of pain, agitation-sedation and delirium assessments in Chinese intensive care units.
      Physicians’ rating of their own diagnostic performance is unfortunately hard to find, much less correlate with actual results. However, with the wide availability and astounding advances in imaging and endoscopic techniques, and the increasing spectrum and precision of laboratory tests, many physicians are quite likely to perceive current diagnostic accuracy as being nearly infallible. However, the reality is quite different, as data on diagnostic errors, overdiagnosis, observer disagreement, diagnosis by serendipity, and “no diagnosis” reveal.

      Diagnostic Errors

      Except for patient and physician surveys, information on diagnostic errors remains dependent mostly on claims data, patients’ stories, physicians’ reports, and autopsy results, which reflect post hoc data often centered on extreme cases and subject to hindsight bias. A comparison of clinical diagnosis versus autopsy findings in 400 cases over 4 eras reveals ∼10% of misdiagnosis that was not reduced over time by the introduction of new diagnostic procedures.
      • Kirch W
      • Schafii C.
      Misdiagnosis at a university hospital in 4 medical eras. Report on 400 cases.
      Other research supports that despite sophisticated technology and constantly improving knowledge base, errors, including delayed diagnosis, erroneous diagnosis, and missed diagnosis, remain common, and is estimated at ∼15% and are often associated with adverse patient outcomes.
      • Graber ML.
      The incidence of diagnostic error in medicine.

      Overdiagnosis

      The identification of findings that are true-positive but unlikely to be of clinical significance or turn out after a waste of time and resources to be false-positive constitute the problem of overdiagnosis, a common occurence with adverse consequences for patients, physicians, and health care systems.

      Observer Variation

      For a diagnosis to be reliable, interobserver variation (and intraobserver variation) need to demonstrate a relatively high degree of agreement quantitated by the kappa statistic (κ) where values near 0 indicate pure chance and a value of 1 indicates perfect agreement. However, when κ was tested for physical examination findings, research often indicates an agreement <0.6, which is moderate at best, often variable, and therefore, a constant potential source of error that needs to be taken into account. For example, the κ-statistic for jugular venous pressure determination was 0.08-0.71; for pulmonary crackles 0.21-0.65; for rebound abdominal tenderness 0.25; and for Babinski sign 0.17-0.55.
      • McGee S.
      Evidence-based physical diagnosis.
      The same applies to interpretation of myriad imaging studies or pathology reports, detracting from the “absolute” credibility often assumed by clinicians who receive such reports.
      • Amer SAKS
      • Li TC
      • Bygrave C
      • et al.
      An evaluation of the inter-observer and intra-observer variability of the ultrasound diagnosis of polycystic ovaries.

      Serendipity

      Serendipity in diagnosis can be defined as obtaining an unexpected explanation of the patient's complaints by test(s) ordered for other indications. Although undoubtedly quite common in every setting, diagnosis by serendipity is understudied and underreported, excepting occasional case reports and small series limited to a single condition.
      • Oshmyanski AR
      • Mahammedi A
      • Dackiw A
      • et al.
      Serendipity in the diagnosis of pheochromocytoma.
      Serendipity still yields an accurate diagnosis, but its achievement by chance undermines physicians’ accomplishments.

      Lost and Found

      In 1 study, important diagnoses of treatable infectious diseases, even malignant conditions, were made, but the crucial report arrived after the patient was discharged and received no attention.
      • Gandhi TK.
      Fumbled handoffs: one dropped ball after another.

      No Diagnosis Can Be Reached

      It is intriguing to find out that in 2 relatively recent series of patient with fever of unknown origin (FUO) studied at dedicated centers, as many as 51%-53% of patients could not be diagnosed, and this percentage is paradoxically increasing.
      • Bleeker-Rovers CP
      • Vos FJ
      • de Kleijn EMHA
      • et al.
      A prospective multicenter study on fever of unknown origin. The yield of a structured diagnostic protocol.
      This unexpected observation also applies to primary care, where patients dubbed as exhibiting medically unexplained symptoms (MUS) are prevalent.
      • Kirmayer LJ
      • Groleau D
      • Looper KJ
      • Dao MD.
      Explaining medically unexplained symptoms.
      Prospective longitudinal cohort studies of MUS are scarce, and the few published ones focus on psychiatric morbidity and mental distress. However, some of these patients are subsequently given a bona-fide somatic diagnosis. For example, the mean time to diagnosis of lupus and related autoimmune diseases was 6 years 11 months, with 24% reporting that diagnosis took >10 years and 76% reporting misdiagnosis including frequent nonorganic and MUS erroneous labeling.
      • Sloan M
      • Harwood R
      • Sutton S
      • et al.
      Medically unexplained symptoms: a mixed methods study of diagnostic, symptom and support experiences of patients with lupus and related systemic autoimmune diseases.

      Conclusions

      The prevailing view of extreme effectiveness of diagnosis tends to disregard significant errors and limitations of the diagnostic process that have not been eliminated by the constantly improving scientific knowledge and technological capabilities. Diagnostic successes are readily published and seem ubiquitous, while unfortunate delays, serious errors, and chronically undiagnosed patients known to every clinician remain in the realm of personal knowledge. Research efforts are also nominal: out of a meager 52 published articles on “diagnostic failure” over the last 20 years, 31 (60%) were devoted to interpretation of biopsies, and none examined the problem as such (PubMed search).
      How do these limitations translate into clinical practice? First, adopting an attitude of humility and skepticism is one important key. Double-checking that all significant history and laboratory and imaging results can be explained by the postulated diagnosis; reflecting on the diagnosis to ensure it was not a diagnosis of convenience (eg, assuming that readily identifiable gallbladder stones caused the patients’ abdominal pain without looking further); looking up databases to confirm finer points; and keeping an open mind to alternatives are essential routines. Second, increasing research efforts in different settings, centered on identifying the more common causes of errors in diagnosis, is highly warranted. A prospective design, hard as it may be to implement, is desirable, and important insights might be achieved on faulty data collection, excessive dependence on heuristics in complex problems, and failure to acknowledge limitations of imaging techniques (assuming they are more sensitive and specific than studies demonstrate). The bottom line is that clinicians’ diagnostic accuracy remains imperfect and could be improved provided its limitations are acknowledged

      References

        • Handwashing Liaison Group
        Hand washing. A modest measure – with big effects.
        BMJ. 1999; 318: 686
        • Schattner A
        • Gilad A.
        Primary care physicians’ awareness and implementation of screening guidelines for colorectal cancer.
        Prev Med. 2002; 35: 447-452
        • Zeuner R
        • Frosch DL
        • Kuzemchak MD
        • Politi MC.
        Physicians’ perceptions of shared decision-making behaviours: a qualitative study demonstrating the continued chasm between aspirations and clinical practice.
        Health Expect. 2014; 18: 2465-2476
        • Chen K
        • Yang YL
        • Li HL
        • et al.
        A gap existed between physicians’ perceptions and performance of pain, agitation-sedation and delirium assessments in Chinese intensive care units.
        BMC Anesthesiol. 2021; 21: 61https://doi.org/10.1186/s12871-021-01286-w
        • Kirch W
        • Schafii C.
        Misdiagnosis at a university hospital in 4 medical eras. Report on 400 cases.
        Medicine (Baltimore). 1996; 75: 29-40
        • Graber ML.
        The incidence of diagnostic error in medicine.
        BMJ Qual Saf. 2013; : ii21-ii27https://doi.org/10.1136/bmjqs-2012-001615
        • McGee S.
        Evidence-based physical diagnosis.
        3rd ed. Elsevier Saunders, Philadelphia, PA2012: 29-39
        • Amer SAKS
        • Li TC
        • Bygrave C
        • et al.
        An evaluation of the inter-observer and intra-observer variability of the ultrasound diagnosis of polycystic ovaries.
        Hum Reprod. 2002; 17: 1616-1622
        • Oshmyanski AR
        • Mahammedi A
        • Dackiw A
        • et al.
        Serendipity in the diagnosis of pheochromocytoma.
        J Comput Assist Tomogr. 2013; 37: 820-823
        • Gandhi TK.
        Fumbled handoffs: one dropped ball after another.
        Ann Intern Med. 2005; 142: 352-358
        • Bleeker-Rovers CP
        • Vos FJ
        • de Kleijn EMHA
        • et al.
        A prospective multicenter study on fever of unknown origin. The yield of a structured diagnostic protocol.
        Medicine. 2007; 86: 26-38
        • Kirmayer LJ
        • Groleau D
        • Looper KJ
        • Dao MD.
        Explaining medically unexplained symptoms.
        Can J Psychiatry. 2004; 49: 663-672
        • Sloan M
        • Harwood R
        • Sutton S
        • et al.
        Medically unexplained symptoms: a mixed methods study of diagnostic, symptom and support experiences of patients with lupus and related systemic autoimmune diseases.
        Rheumatol Adv Pract. 2020; 4: rkaa006https://doi.org/10.1093/rap/rkaa006