Advertisement

Evaluation of Outcomes From a National Patient-initiated Second-opinion Program

  • Ashley N.D. Meyer
    Correspondence
    Requests for reprints should be addressed to Ashley N.D. Meyer, PhD, Michael E. DeBakey Veterans Affairs Medical Center, Center for Innovations in Quality, Effectiveness and Safety (152), 2002 Holcombe Boulevard, Houston, TX 77030.
    Affiliations
    Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center, Houston, Tex

    Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, Tex
    Search for articles by this author
  • Hardeep Singh
    Affiliations
    Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center, Houston, Tex

    Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, Tex
    Search for articles by this author
  • Mark L. Graber
    Affiliations
    RTI International, Research Triangle Park, NC

    SUNY Stony Brook School of Medicine, New York, NY
    Search for articles by this author

      Abstract

      Background

      We examined outcomes of patient-initiated second opinions provided by a national second-opinion program.

      Methods

      We independently examined data collected from January 1, 2011 to December 31, 2012 from a second-opinion program (Best Doctors, Inc.) that allows employee-beneficiaries to request free second opinions. Clinical intake included ascertaining why patients sought second opinions and acquiring patients' complete medical records. Trained physicians summarized the cases; identified key, unresolved clinical questions; and forwarded the cases to expert specialists who provided independent assessments and recommendations. Second opinions were discussed with and returned to patients for review with their physicians. Nurses determined whether second opinions confirmed, clarified, or changed initial diagnoses and treatments, and physicians estimated their clinical impact. Patient satisfaction also was surveyed.

      Results

      A total of 6791 patient-initiated second opinions were completed across medical specialties. Patients primarily sought second opinions for help choosing treatment options (41.3%) and for diagnostic concerns (34.8%). Second opinions often resulted in changes in diagnosis (14.8%), treatment (37.4%), or changes in both (10.6%). Clinical impact was estimated as moderate/major in 20.9% of cases for diagnosis and 30.7% of cases for treatment. Changes in diagnoses and/or treatments and clinical impact varied across medical specialties. In patients surveyed (n = 2683), most (94.7%) were satisfied with the experience, but fewer (61.2%) planned to follow the recommendations.

      Conclusions

      Patient-initiated second opinions led to recommended changes in diagnosis for about 15% and in treatment for about 37% of participants. Further evaluation is needed to determine whether this impacts clinical outcomes, such as the reduction of diagnosis and treatment errors.

      Keywords

      Clinical Significance
      • In a large-scale evaluation of a national second-opinion program including multiple specialties, patient-initiated second opinions led to recommended changes in diagnosis for about 15% of participants, in treatment for about 37% of participants, and changes in both for more than 10%.
      • The proportion of resulting changes in diagnosis and treatment differed widely by specialty.
      • Treatment changes occurred more often than diagnostic changes in every specialty.
      Patients seek second medical opinions for a variety of reasons.
      • Landro L.
      What if the doctor is wrong? Some cancers, asthma, other conditions can be tricky to diagnose, leading to incorrect treatments.
      • Mustafa M.
      • Bijl M.
      • Gans R.
      What is the value of patient-sought second opinions?.
      Some patients seek advice because their symptoms remain undiagnosed. Others are given diagnoses, but their symptoms persist, they have doubts about their diagnoses, or they hope their diagnoses are incorrect, especially when the diagnoses carry substantial risks of major morbidity or mortality.
      • Mellink W.A.
      • Dulmen A.M.
      • Wiggers T.
      • Spreeuwenberg P.M.
      • Eggermont A.M.
      • Bensing J.M.
      Cancer patients seeking a second surgical opinion: results of a study on motives, needs, and expectations.
      While seeking second opinions, patients are looking for more information about their diseases or treatment plans,
      • Wijers D.
      • Wieske L.
      • Vergouwen M.D.
      • Richard E.
      • Stam J.
      • Smets E.M.
      Patient satisfaction in neurological second opinions and tertiary referrals.
      • Tattersall M.H.
      • Dear R.F.
      • Jansen J.
      • et al.
      Second opinions in oncology: the experiences of patients attending the Sydney Cancer Centre.
      for confirmation that their diagnoses or treatment plans are correct,
      • Sutherland L.R.
      • Verhoef M.J.
      Patients who seek a second opinion: are they different from the typical referral?.
      • Sutherland L.R.
      • Verhoef M.J.
      Why do patients seek a second opinion or alternative medicine?.
      or they dislike their treatment options. In one large national sample, one-fifth of patients who saw a doctor in the past year sought a second opinion,
      • Wagner T.H.
      • Wagner L.S.
      Who gets second opinions?.
      and this fraction exceeds 50% in patients diagnosed with cancer.
      • Hewitt M.
      • Breen N.
      • Devesa S.
      Cancer prevalence and survivorship issues: analysis of the 1992 National Health Interview Survey.
      Many specialty practices already devote a significant fraction of their workload to providing second opinions,
      • Wagner T.H.
      • Wagner L.S.
      Who gets second opinions?.
      a trend likely to increase as patients become more proactive in their care.
      The impact of patient-initiated second opinions on outcomes is unclear but is a worthy area to investigate because they could be a strategy to reduce errors in diagnosis
      • Graber M.L.
      • Kissam S.
      • Payne V.L.
      • et al.
      Cognitive interventions to reduce diagnostic error: a narrative review.
      or treatment. Literature from second reviews in pathology and radiology shows that although second opinions differ from the first in a relatively small fraction (generally 2%-20%) of cases,
      • Swapp R.E.
      • Aubry M.C.
      • Salomão D.R.
      • Cheville J.C.
      Outside case review of surgical pathology for referred patients: the impact on patient care.
      • Goddard P.
      • Leslie A.
      • Jones A.
      • Wakeley C.
      • Kabala J.
      Error in radiology.
      • Jackson S.L.
      • Taplin S.H.
      • Sickles E.A.
      • et al.
      Variability of interpretive accuracy among diagnostic mammography facilities.
      these changes can have a major impact on the prognoses or treatments recommended. In contrast, less is known about the impact of patient-initiated second opinions for general medical and surgical concerns.
      • Payne V.L.
      • Singh H.
      • Meyer A.N.D.
      • Levy L.
      • Harrison D.
      • Graber M.L.
      Patient-initiated second opinions: systematic review of characteristics and impact on diagnosis, treatment, and satisfaction.
      We thus evaluated outcomes from a nationally administered second-opinion program that allows employee-beneficiaries to request second opinions from expert specialists. Outcomes included the frequency of changes in diagnostic and therapeutic recommendations, their estimated clinical impact, and patient satisfaction.

      Methods

      Second-Opinion Program Process

      We evaluated second opinions provided to patients during 2011 and 2012 by a nationally administered program (Best Doctors, Inc.) that allows covered employees to request second opinions at no additional cost. After an enrolled patient requests a second opinion, clinical program staff interview the patient and determine the patient's reason for the request, perform a clinical intake, and obtain the patient's complete medical record, including notes, tests (laboratory, pathology, and imaging), and procedures that have been performed (Figure 1). All imaging and pathology are rereviewed independently, and in appropriate cases unstained biopsy material is restained and analyzed. The collected information and the reanalyzed imaging and pathology are then reviewed by trained program physicians, who aim to generate a comprehensive clinical summary and a list of 8-15 key, unresolved clinical questions that need to be answered by the expert consultant. The summary, key questions, and all of the available diagnostic data are then forwarded to a specialist consultant to provide the second opinion on both the patient's diagnosis and treatment. The specialist reviews the entire case and questions posed and creates a detailed “expert's report” containing his or her impressions and recommendations regarding the likely diagnosis, additional tests suggested, if any, and recommended treatment. This report is returned to the program clinical staff, who review and grade the specialist's report and then forward it to the patient, along with an encrypted USB drive containing all of his or her collected medical information. A physician or nurse also reviews the expert report with the patient and outlines next recommended steps. Additionally, the patient is advised to review the findings with his or her treating physician. For patients in critical care, collecting all of the records/tests/imaging takes approximately 1 day, and expert reports are delivered 2 to 3 days after that. For ambulatory patients, data collection takes 2-4 weeks, and expert reports are delivered 7-10 days later.
      Figure thumbnail gr1
      Figure 1Overview of Best Doctors patient-initiated second-opinion process. (1) Patient initiates second opinion. (2) Program staff perform a clinical intake and obtain the patient's entire medical record and diagnostic tests performed. All imaging and pathology are reviewed. (3) Collected information is forwarded to trained physicians, who do a comprehensive analysis and produce a clinical summary of the case and key, unresolved clinical questions. (4) This summary, along with all of the diagnostic data and key questions, are forwarded to expert specialists, who synthesize the information and create a detailed “expert's report” containing impressions and recommendations regarding a clinical plan (including additional tests to get done and optimal treatment to pursue). (5) The expert's report and recommended clinical plan are returned to the program office, where the report is graded. (6) The expert's report is discussed with the patient and sent to the patient with advice to review the findings with his or her treating physician.
      The physicians and expert specialists are identified by nominations from the programs' existing physicians. Biennially, existing and nominated physicians are evaluated confidentially by other participating physicians and are invited to participate if they obtain strong evaluations from their peers, have an active medical license, have no disciplinary record, and actively see patients. The process aims to identify the best 5% of specialist consultants in the United States.

      Outcomes of the Second-Opinion Program

      The program collects information on the frequency of second opinions initiated, the reasons they were sought, changes in diagnosis and treatment, estimated clinical impact, and patient satisfaction. The reasons patients sought the second opinions are collected during the intake process, and patients are told to choose one from multiple options. After a second opinion is rendered, trained nurses compare experts' diagnostic and treatment recommendations with the initial diagnoses and treatment recommendations, classifying each second opinion as confirming, clarifying, or changing the initial diagnoses and/or treatments (Table shows classification definitions and examples). Additionally, trained program physicians estimate the magnitude of the clinical impact the second opinion might have on diagnosis and/or treatment (rated as none, minor, moderate, or major; Table shows classification definitions and examples). One to two weeks after receiving their report, patients are invited to complete satisfaction surveys through e-mail (Supplementary Table 1, available online).
      TableDefinitions and Examples of Classifications of Changes and Estimated Impact for Both Diagnosis and Treatment
      DefinitionExample
      Changes in Diagnosis
       Confirm: The export report created after the second opinion affirms the accuracy of the treating physician's diagnosis/diagnostic plan or recommends only trivial modifications.The diagnosis and staging of the patient's cervical cancer is correct.
       Clarify: The export report created after the second opinion refines the treating physician's diagnosis in a meaningful way or outlines next steps in establishing a diagnosis.The patient's nonspecific back pain is due to lumbar spondylosis (arthritis of the spine).
       Change: The export report created after the second opinion refines the treating physician's diagnosis in a way that impacts management offers a different diagnosis, or (for cases with pathology) provides a different pathologic interpretation.The patient does not have cervical cancer; she has metastatic colon cancer.
      Changes in Treatment
       Confirm: The export report created after the second opinion affirms the appropriateness of the treating physician's therapeutic plan or recommends only trivial modifications.The current chemotherapy regimen for the patient's cervical cancer is a first-line regimen.
       Clarify: The export report created after the second opinion directs the patient in the selection of multiple proposed therapeutic options or proposes some fairly small alterations in the treatment plan.The current regimen of 5-fluorouracil plus cisplatin is reasonable but would recommend administering cisplatin weekly, as this has a more favorable toxicity profile.
       Change: The export report created after the second opinion recommends significant changes to the treating physician's plan.The current regimen of 5-fluorouracil plus cisplatin is not appropriate, and the expert would recommend treatment with oxaliplatin and etoposide.
      Estimated Clinical Impact of Second Opinion on Diagnosis
       No change: The treating physician's differential diagnosis is broad, relevant and within context of the patient's comorbidities; and the second opinion confirms that the working diagnoses are accurate and clear.An established diagnosis of breast cancer is confirmed by pathology review and staging is deemed correct by the expert.
       Minor change: Second opinion modestly broadens the treating physician's differential diagnosis or suggests additional diagnostic studies of modest importance or refines the working diagnosis to be more specific, relevant, or clear.An established diagnosis of breast cancer is confirmed by pathology review, but the staging is refined (no change in therapy, but change in prognosis), or a bone density scan is recommended prior to initiating therapy with tamoxifen.
       Moderate change: Second opinion significantly broadens the treating physician's differential diagnosis to include important possibilities not considered or suggests additional important diagnostic studies.An established diagnosis of breast cancer is confirmed but staging is significantly changed (so as to affect treatment) or genetic testing is recommended that has a significant chance of affecting therapy.
       Major change: Second opinion significantly broadens the treating physician's differential diagnosis to include critical possibilities not considered and suggests additional important diagnostic studies or changes an incorrect diagnosis.A diagnosis of ductal carcinoma in situ is changed to invasive carcinoma and previously missed lymphovascular invasion is identified upon pathology review.
      Estimated Clinical Impact of Second Opinion on Treatment
       No change: Second opinion confirms that the current treatment plan is accurate, clear and complete.The expert comments that a patient's regimen for breast cancer treatment is appropriate, and also outlines some reasonable alternative regimens.
       Minor change: Second opinion clarifies the current treatment plan; or recommends additional therapeutic modalities that will modestly enhance the patient's current care.The expert suggests a different chemotherapy dosing schedule to lower toxicity for a member with breast cancer, but the overall regimen remains the same.
       Moderate change: Second-opinion recommends additional therapeutic modalities that will significantly enhance the patient's current care.A patient with metastatic breast cancer is recommended to start monthly bisphosphonate therapy, or the expert provides a patient with critical information that enables her to decide between lumpectomy and prophylactic bilateral mastectomy.
       Major change: Second opinion recommends additional therapeutic modalities that will dramatically enhance the patient's current care or corrects a flawed treatment plan.After a patient is determined upon pathology review to have invasive breast cancer rather than ductal carcinoma in situ, the expert recommends a significantly different and more aggressive treatment regimen.

      Independent Analyses

      The study was exempted from institutional review board approval. Data from all patient-initiated second opinions completed between January 1, 2011 and December 31, 2012 were aggregated and independently analyzed by a research team not associated with the program. Percentages and frequencies were used to summarize most data, and a McNemar's test was used to compare the percentage of second opinions resulting in changes in diagnosis with the percentage leading to changes in treatment. A McNemar's test also was used to compare the percentage of second opinions estimated to have a moderate/major impact on diagnosis with the percentage estimated to have a moderate/major impact on treatment. Additionally, across specialties, we compared the percentage of second opinions resulting in changes in diagnosis and in treatment, as well as the percentage of cases with moderate/major impact on diagnosis and on treatment, using analysis of variance tests with planned deviant contrasts (ie, contrasts in which the percentage in each specialty is compared with the mean percentage). Analyses were done using IBM SPSS Statistics 22 and considered significant when P < .05. Individually identifiable patient and provider information was not sent to the independent study team.

      Results

      Frequency of Second Opinions

      Over the 2-year study period, the number of insured lives eligible for the program was 6,605,675 (3,082,394 insured subjects plus their insured dependents). From this group, 6791 second opinions were requested and completed (0.1%). Although these data include patients from all over the United States, we did not have access to data on which facilities in the United States patients obtained their first opinions from.
      Of all the requests received, almost half were accounted for within 5 of the 34 represented medical specialties (Supplementary Tables 2 and 3, available online). Requests for second opinions in Orthopedic Surgery were the most common (n = 1195), followed by Medical Oncology and Hematology (n = 588). Requests for second opinions in Family Medicine and in Sleep Medicine were the least common (n = 4 and 5, respectively).

      Reasons for Seeking Second Opinions

      Patients' reasons for requesting second opinions are presented in Figure 2. More than half were related to treatment questions, including the need to decide among different treatment options (41.3%) or whether to proceed with recommended surgery (18.0%). Questions relating to diagnosis were the motivating factor in 34.8%, including patients whose symptoms had not improved (22.5%), who remained undiagnosed (6.3%), or who had residual questions about their diagnoses (6.0%).
      Figure thumbnail gr2
      Figure 2Reasons patients sought second opinions, collected during clinical intake. Reasons were mutually exclusive.

      Changes Resulting From Second Opinions in Diagnosis and Treatment

      Changes in diagnoses and treatments are presented in Figure 3. The majority of reviews resulted in confirmation (56.8% and 26.4% for diagnostic and treatment opinions, respectively) or clarification (17.0% and 26.9%) of the original opinion for both diagnosis and treatment-related issues. However, there were many cases in which the diagnosis (14.8%) or treatment (37.4%) was changed. Furthermore, 10.6% of cases had changes in both diagnosis and treatment. Treatment changes, however, were recommended more often than diagnosis changes (P < .001).
      Figure thumbnail gr3
      Figure 3Changes in diagnoses and treatments from second opinion in 6791 cases. Percentages of patients whose second opinions led to changes, clarifications, or confirmations of diagnoses and of treatments, as assessed by program clinical staff.
      The percentage of cases with changes differed by specialty for both diagnosis (F33, 6757 = 6.40; P < .001) and treatment (F33, 6757 = 6.40; P < .001). For diagnosis, second opinions in Cardiovascular Disease (n = 359; 9.2%), Medical Oncology and Hematology (n = 588; 5.1%), Surgical Oncology (n = 58; 3.4%), and Urology (n = 298; 7.7%) resulted in significantly fewer changes than average (mean 14.8%), whereas second opinions in the specialties of Anesthesiology (n = 39; 35.9%), Gastroenterology (n = 473; 23.7%), Neurology (n = 577; 22.5%), and Rheumatology (n = 240; 26.3%) resulted in significantly more changes than average (Supplementary Table 2, available online).
      For treatment, second opinions in General Surgery (n = 113; 21.2%), Medical Oncology and Hematology (n = 588; 27.0%), Surgical Oncology (n = 58; 19.0%), and Urology (n = 298; 28.5%) resulted in significantly fewer changes than average (mean 37.4%). Conversely, second opinions in the specialties of Allergy and Immunology (n = 29; 58.6%), Anesthesiology (n = 39; 69.2%), Gastroenterology (n = 473; 49.3%), Neurological Surgery (n = 259; 42.5%), Obstetrics and Gynecology (n = 320; 42.5%), Otolaryngology (n = 233; 44.2%), Physical Medicine and Rehabilitation (n = 458; 41.3%), and Rheumatology (n = 240; 46.7%) resulted in significantly more changes than average (Supplementary Table 3, available online).

      Impact of Second Opinions on Diagnosis and Treatment

      The clinical impact of the opinions for both diagnosis and treatment, although most commonly estimated to be minor, was moderate or major in approximately one-third of cases (Figure 4). Specifically for diagnosis, the second opinions were estimated to have minor clinical impact in 46.3% of cases, but moderate impact in 18.2% and major impact in 2.7% of cases. For treatment, the second opinions were estimated to have minor clinical impact in 50.1% of cases, but moderate impact in 26.5% and major impact in 4.2% of cases. Overall, the clinical impact of the second opinion was estimated to have a moderate/major impact on treatment more often than on diagnosis (P < .001).
      Figure thumbnail gr4
      Figure 4Estimated clinical impact of second opinions in 6791 cases, as assessed by program physicians.
      The percentage of cases with moderate/major clinical impact differed by specialty for both diagnosis (F33, 6757 = 3.44; P < .001) and treatment (F33, 6757 = 3.27; P < .001). For diagnosis, second opinions in General Surgery (n = 113; 9.8%), Ophthalmology (n = 140; 12.1%), and Radiation Oncology (n = 42; 2.4%) resulted in significantly fewer estimates of moderate/major clinical impact than average (mean 20.9%); conversely, second opinions in the specialties of Critical Care/Pulmonary Medicine (n = 123; 36.6%), Gastroenterology (n = 473; 26.9%), Infectious Diseases (n = 59; 32.2%), Neurology (n = 577; 27.2%), and Obstetrics and Gynecology (n = 320; 27.5%) resulted in significantly more estimates of moderate/major clinical impact than average (Supplementary Table 2, available online gives a full list of specialties and their outcomes).
      For treatment, second opinions in Cardiovascular Disease (n = 359; 24.6%), General Surgery (n = 113; 18.6%), Internal Medicine (n = 87; 16.1%), Neurology (n = 577; 24.3%), Ophthalmology (n = 140; 22.1%), and Physical Medicine and Rehabilitation (n = 458; 25.1%) resulted in significantly fewer estimates of moderate/major clinical impact than average (mean 30.7%), whereas second opinions in the specialties of Colon and Rectal Surgery (n = 24; 50.0%), Medical Oncology and Hematology (n = 588; 37.9%), Obstetrics and Gynecology (n = 320; 40.9%), and Thoracic Surgery (n = 23; 52.1%) resulted in significantly more estimates of moderate/major clinical impact than average (Supplementary Table 3, available online).

      Satisfaction of Second Opinions

      In the 2-year period, the program sent out 6248 surveys to the patients with current e-mail addresses on file and obtained 2683 responses (42.9% response rate). When surveyed at the completion of the second-opinion process, 94.7% of patients were satisfied with the experience, and 89.6% said their questions were answered (Supplementary Table 1, available online). Most participating patients (83.6%) had discussed the findings with their physicians, but only 61.2% agreed or strongly agreed that they were going to follow the second-opinion recommendations. Last, 87.3% were more confident in their diagnosis or treatment choice.

      Discussion

      Patient-initiated second opinions led to recommended changes in diagnosis for about 15% of participants, changes in treatment for about 37% of participants, and changes in both diagnosis and treatment for more than 10%. Additionally, the second opinions were estimated to have moderate or major clinical impact on patients' diagnoses in more than 20% of cases and on the patients' treatments in more than 30%. Second opinions within certain specialties were more likely to result in changes in diagnosis and treatment than others. Similarly, second opinions within certain specialties were more likely to have a moderate or major clinical impact than others. These findings suggest significant diagnostic and treatment variability in real-world clinical practice and are consistent with findings using trained standardized patients.
      • Peabody J.W.
      • Luck J.
      • Jain S.
      • Bertenthal D.
      • Glassman P.
      Assessing the accuracy of administrative data in health information systems.
      Second opinions have been suggested as a strategy to prevent diagnostic and treatment errors.
      • Graber M.L.
      • Kissam S.
      • Payne V.L.
      • et al.
      Cognitive interventions to reduce diagnostic error: a narrative review.
      The apparent rate of diagnostic variation identified in this study, roughly 15%, is in line with estimates of diagnostic errors from a wide range of different research approaches.
      • Graber M.L.
      The incidence of diagnostic error in medicine.
      However, further evaluations, including long-term follow-up of patient outcomes, would be needed to determine whether the second opinions were indeed correct and whether they reduced the risk of diagnostic error. Studies using follow-up evaluation in pathology have found that the second opinion itself is incorrect in 7%-15% of cases, and in some of these instances the correct diagnosis was the first one.
      • Swapp R.E.
      • Aubry M.C.
      • Salomão D.R.
      • Cheville J.C.
      Outside case review of surgical pathology for referred patients: the impact on patient care.
      • Park J.H.
      • Kim H.K.
      • Kang S.W.
      • et al.
      Second opinion in thyroid fine-needle aspiration biopsy by the Bethesda system.
      To our knowledge, this is the largest evaluation of second opinions across several specialties. The number of requests for second opinions varied substantially between different specialties, ranging from only a handful of requests in family medicine and in sleep medicine to more than 1000 requests for orthopedic issues. Additionally, the proportion of resulting changes in diagnosis and treatment differed widely by specialty, although treatment changes occurred more often than diagnostic changes in every specialty.
      Our findings have important implications for future use of second opinions. First, there is a growing appreciation of diagnostic error. Recent estimates in US ambulatory care settings suggest that 1 in 20 patients will experience a diagnostic error annually.
      • Singh H.
      • Meyer A.N.
      • Thomas E.J.
      The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations.
      Second, patients are increasingly becoming interested in becoming active partners in their care.
      • Davis Giardina T.
      • Menon S.
      • Parrish D.E.
      • Sittig D.F.
      • Singh H.
      Patient access to medical records and healthcare outcomes: a systematic review.
      • Walker J.
      • Leveille S.G.
      • Ngo L.
      • et al.
      Inviting patients to read their doctors' notes: patients and doctors look ahead: patient and physician surveys.
      Patient engagement changes the dynamic relationship between patients and their providers, such that patients want and expect greater transparency and more information about their conditions, and are more willing to question medical advice.
      • Conway J.
      • Johnson B.
      • Edgman-Levitan S.
      • et al.
      Partnering with Patients and Families to Design a Patient- and Family-Centered Health Care System: A Roadmap for the Future.
      • McDonald K.M.
      • Bryce C.
      • Graber M.L.
      The patient is in: patient involvement strategies for diagnostic error mitigation.
      • Greene A.
      Participatory diagnosis.
      Although patient satisfaction with the program was very high, consistent with existing evidence,
      • Payne V.L.
      • Singh H.
      • Meyer A.N.D.
      • Levy L.
      • Harrison D.
      • Graber M.L.
      Patient-initiated second opinions: systematic review of characteristics and impact on diagnosis, treatment, and satisfaction.
      patients did not always follow the second opinion. This suggests that in these cases the patients may be either trusting their initial advice or somehow integrating the advice obtained from their original providers with the new information. Additional research could determine why patients decide to choose one opinion versus the other. For example, some patients might be more engaged in their own care and independently evaluate the wisdom of the second-opinion advice provided.
      Our analyses and its application have several limitations because of our focus on a single second-opinion program. First, the grading of whether diagnoses or treatment recommendations changed and the impact of the second opinions was conducted by internal program staff, allowing for the potential of bias in these evaluations. However, the outcome data were evaluated independently. Second, we did not have data from nonresponders, who might be less satisfied with the program. Third, the second opinions provided in this program were free of charge to the patients, and implications of our study might be limited with respect to other patient populations. Fourth, because patients self-selected second opinions, the frequency of disagreement between the initial advice and the second opinion in this population could be higher than in the general patient population. Fifth, the extensive data gathering, case synthesis, and review process may make replication difficult in other programs. However, we are unaware of a similar large-scale evaluation of any patient-initiated second-opinion program involving multiple specialties, and our study might be useful for other programs geared to help patients with diagnosis- and treatment-related decisions. Last, the study did not include long-term follow-up to confirm whether, in fact, the second opinion was correct, or to verify the impact of the changes made on patient outcomes.
      In conclusion, patient-initiated second opinions led to recommended changes in diagnosis or treatment for a large portion of participants. Second opinions may identify variation in real-world clinical practice, potentially serving as a way to discover errors in diagnosis or in the formulation of treatment plans. Further evaluation, especially data on long-term follow-up of patients, is needed to determine whether second opinions impact clinical outcomes, including the reduction of harm associated with diagnostic or treatment errors.

      Acknowledgment

      The authors thank Best Doctors, Inc. for providing access to their data and for help in verifying the details of their second-opinion process.

      Supplementary Data.

      Supplementary Table 1Patient Satisfaction Survey and Responses
      Survey ItemNo. of Patient ResponsesStrongly DisagreeDisagreeNeither Disagree Nor AgreeAgreeStrongly Agree
      I am satisfied with the experience.268120 (0.7)35 (1.3)88 (3.3)603 (22.5)1935 (72.2)
      The second opinion answered my questions.268239 (1.5)59 (2.2)181 (6.7)778 (29.0)1625 (60.6)
      I have discussed the results with my physician.268143 (1.6)74 (2.8)322 (12.0)745 (27.8)1497 (55.8)
      I plan to follow the recommendations.268052 (1.9)84 (3.1)903 (33.7)708 (26.4)933 (34.8)
      I am more confident in my diagnosis or treatment choice.268240 (1.5)64 (2.4)237 (8.8)759 (28.3)1582 (59.0)
      Values are presented as number (percentage).
      Supplementary Table 2Changes and Impact on Diagnosis for All Specialties
      SpecialtyTotal Cases (n)Diagnosis Changes (%)Impact of Evaluation on Diagnosis (%)
      ChangedConfirmed or ClarifiedPendingNoneMinorModerateMajorPending
      Allergy and Immunology2920.765.513.86.955.220.76.910.3
      Anesthesiology/Pain Medicine3935.961.52.638.546.210.35.10.0
      Cardiovascular Disease3599.281.19.724.057.413.11.93.6
      Colon and Rectal Surgery2416.770.812.520.862.512.54.20.0
      Critical Care Medicine-Pulmonary Medicine12319.557.722.811.449.632.54.12.4
      Dermatology10016.070.014.032.037.022.04.05.0
      Endocrinology and Metabolism25111.677.710.828.745.418.72.05.2
      Family Medicine425.075.00.025.075.00.00.00.0
      Gastroenterology47323.758.817.518.850.323.53.44.0
      General Surgery1139.785.05.343.438.18.01.88.8
      Hand Surgery697.276.815.936.240.613.01.48.7
      Hepatology575.384.210.521.149.114.07.08.8
      Infectious Disease5922.061.016.911.947.528.83.48.5
      Internal Medicine8720.754.025.319.544.821.80.013.8
      Medical Genetics205.080.015.035.040.020.00.05.0
      Medical Oncology and Hematology5885.191.73.232.543.018.53.22.7
      Nephrology545.683.311.116.750.025.93.73.7
      Neurologic Surgery25917.873.09.331.346.714.72.74.6
      Neurology57722.562.415.121.546.623.63.64.7
      Obstetrics and Gynecology32014.773.112.223.442.523.44.16.6
      Ophthalmology14012.978.68.635.049.311.40.73.6
      Orthopedic Surgery119513.877.29.031.743.516.02.56.3
      Otolaryngology23317.276.06.930.542.518.03.45.6
      Pediatric Specialist41114.476.49.221.752.615.63.96.3
      Physical Medicine and Rehabilitation45816.661.621.822.355.717.51.13.5
      Plastic Surgery238.778.313.034.839.117.40.08.7
      Radiation Oncology427.192.90.052.445.22.40.00.0
      Radiology166.393.80.025.062.56.30.06.3
      Rheumatology24026.359.214.627.545.020.82.54.2
      Sleep Medicine50.0100.00.040.040.020.00.00.0
      Surgical Oncology583.493.13.451.732.812.11.71.7
      Thoracic Surgery230.091.38.726.147.817.48.70.0
      Urology2987.782.69.745.334.916.40.33.0
      Vascular Surgery4411.481.86.836.440.918.20.04.5
      All Specialties679114.873.811.427.946.318.22.74.9
      Supplementary Table 3Changes and Impact on Treatment for All Specialties
      SpecialtyTotal Cases (n)Treatment Changes (%)Impact of Evaluation on Treatment (%)
      ChangedConfirmed or ClarifiedPendingNoneMinorModerateMajorPending
      Allergy and Immunology2958.627.613.86.944.831.06.910.3
      Anesthesiology/Pain Medicine3969.230.80.07.761.528.22.60.0
      Cardiovascular Disease35937.352.99.713.457.920.14.54.2
      Colon and Rectal Surgery2441.754.24.20.050.033.316.70.0
      Critical Care Medicine-Pulmonary Medicine12339.039.821.18.952.834.10.83.3
      Dermatology10038.052.010.09.047.034.05.05.0
      Endocrinology and Metabolism25139.451.09.614.348.228.73.65.2
      Family Medicine450.050.00.00.075.025.00.00.0
      Gastroenterology47349.336.814.010.453.729.03.23.8
      General Surgery11321.275.23.525.746.915.92.78.8
      Hand Surgery6937.755.17.218.843.526.14.37.2
      Hepatology5724.663.212.315.854.415.85.38.8
      Infectious Disease5945.842.411.922.037.327.15.18.5
      Internal Medicine8741.443.714.916.152.916.10.014.9
      Medical Genetics2015.065.020.030.040.020.00.010.0
      Medical Oncology and Hematology58827.068.94.118.241.834.23.72.0
      Nephrology5431.557.411.17.453.731.55.61.9
      Neurologic Surgery25942.550.27.312.749.426.67.33.9
      Neurology57742.343.014.713.357.521.72.64.9
      Obstetrics and Gynecology32042.549.77.810.641.632.88.16.9
      Ophthalmology14030.062.97.112.961.420.71.43.6
      Orthopedic Surgery119534.657.48.012.948.326.46.06.4
      Otolaryngology23344.251.54.318.548.123.63.96.0
      Pediatric Specialist41133.859.46.811.451.324.85.47.1
      Physical Medicine and Rehabilitation45841.343.914.814.057.222.92.23.7
      Plastic Surgery2334.847.817.413.047.821.78.78.7
      Radiation Oncology4221.478.60.09.554.833.32.40.0
      Radiology1618.881.30.050.031.312.50.06.3
      Rheumatology24046.743.310.019.248.824.62.94.6
      Sleep Medicine50.0100.00.00.060.040.00.00.0
      Surgical Oncology5819.079.31.725.937.929.35.21.7
      Thoracic Surgery2326.165.28.717.430.439.113.00.0
      Urology29828.563.87.717.846.630.22.33.0
      Vascular Surgery4429.563.66.822.745.527.30.04.5
      All Specialties679137.453.39.314.250.126.54.25.0

      References

        • Landro L.
        What if the doctor is wrong? Some cancers, asthma, other conditions can be tricky to diagnose, leading to incorrect treatments.
        The Wall Street Journal, New York, NYAugust 27, 2012 (Sect. Health)
        • Mustafa M.
        • Bijl M.
        • Gans R.
        What is the value of patient-sought second opinions?.
        Eur J Intern Med. 2002; 13: 445-447
        • Mellink W.A.
        • Dulmen A.M.
        • Wiggers T.
        • Spreeuwenberg P.M.
        • Eggermont A.M.
        • Bensing J.M.
        Cancer patients seeking a second surgical opinion: results of a study on motives, needs, and expectations.
        J Clin Oncol. 2003; 21: 1492-1497
        • Wijers D.
        • Wieske L.
        • Vergouwen M.D.
        • Richard E.
        • Stam J.
        • Smets E.M.
        Patient satisfaction in neurological second opinions and tertiary referrals.
        J Neurol. 2010; 257: 1869-1874
        • Tattersall M.H.
        • Dear R.F.
        • Jansen J.
        • et al.
        Second opinions in oncology: the experiences of patients attending the Sydney Cancer Centre.
        Med J Aust. 2009; 191: 209-212
        • Sutherland L.R.
        • Verhoef M.J.
        Patients who seek a second opinion: are they different from the typical referral?.
        J Clin Gastroenterol. 1989; 11: 308-313
        • Sutherland L.R.
        • Verhoef M.J.
        Why do patients seek a second opinion or alternative medicine?.
        J Clin Gastroenterol. 1994; 19: 194-197
        • Wagner T.H.
        • Wagner L.S.
        Who gets second opinions?.
        Health Aff. 1999; 18: 137-145
        • Hewitt M.
        • Breen N.
        • Devesa S.
        Cancer prevalence and survivorship issues: analysis of the 1992 National Health Interview Survey.
        J Natl Cancer Inst. 1999; 91: 1480-1486
        • Graber M.L.
        • Kissam S.
        • Payne V.L.
        • et al.
        Cognitive interventions to reduce diagnostic error: a narrative review.
        BMJ Qual Saf. 2012; 21: 535-557
        • Swapp R.E.
        • Aubry M.C.
        • Salomão D.R.
        • Cheville J.C.
        Outside case review of surgical pathology for referred patients: the impact on patient care.
        Arch Pathol Lab Med. 2013; 137: 233-240
        • Goddard P.
        • Leslie A.
        • Jones A.
        • Wakeley C.
        • Kabala J.
        Error in radiology.
        Br J Radiol. 2001; 74: 949-951
        • Jackson S.L.
        • Taplin S.H.
        • Sickles E.A.
        • et al.
        Variability of interpretive accuracy among diagnostic mammography facilities.
        J Natl Cancer Inst. 2009; 101: 814-827
        • Payne V.L.
        • Singh H.
        • Meyer A.N.D.
        • Levy L.
        • Harrison D.
        • Graber M.L.
        Patient-initiated second opinions: systematic review of characteristics and impact on diagnosis, treatment, and satisfaction.
        Mayo Clin Proc. 2014; 89: 687-696
        • Peabody J.W.
        • Luck J.
        • Jain S.
        • Bertenthal D.
        • Glassman P.
        Assessing the accuracy of administrative data in health information systems.
        Med Care. 2004; 42: 1066-1072
        • Graber M.L.
        The incidence of diagnostic error in medicine.
        BMJ Qual Saf. 2013; 22: ii21-ii27
        • Park J.H.
        • Kim H.K.
        • Kang S.W.
        • et al.
        Second opinion in thyroid fine-needle aspiration biopsy by the Bethesda system.
        Endocr J. 2012; 59: 205-212
        • Singh H.
        • Meyer A.N.
        • Thomas E.J.
        The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations.
        BMJ Qual Saf. 2014; 23: 727-731
        • Davis Giardina T.
        • Menon S.
        • Parrish D.E.
        • Sittig D.F.
        • Singh H.
        Patient access to medical records and healthcare outcomes: a systematic review.
        J Am Med Inform Assoc. 2014; 21: 737-741
        • Walker J.
        • Leveille S.G.
        • Ngo L.
        • et al.
        Inviting patients to read their doctors' notes: patients and doctors look ahead: patient and physician surveys.
        Ann Intern Med. 2011; 155: 811-819
        • Conway J.
        • Johnson B.
        • Edgman-Levitan S.
        • et al.
        Partnering with Patients and Families to Design a Patient- and Family-Centered Health Care System: A Roadmap for the Future.
        Institutue for Family-Centered Care and Institute for Healthcare Improvement, Bethesda, MD2006
        • McDonald K.M.
        • Bryce C.
        • Graber M.L.
        The patient is in: patient involvement strategies for diagnostic error mitigation.
        BMJ Qual Saf. 2013; 22: ii33-ii39
        • Greene A.
        Participatory diagnosis.
        J Participat Med. 2013; 5: e21