Abstract
Background
Clinical scores may help physicians to better assess the individual risk/benefit of oral anticoagulant therapy. We aimed to externally validate and compare the prognostic performance of 7 clinical prediction scores for major bleeding events during oral anticoagulation therapy.
Methods
We followed 515 adult patients taking oral anticoagulants to measure the first major bleeding event over a 12-month follow-up period. The performance of each score to predict the risk of major bleeding and the physician's subjective assessment of bleeding risk were compared with the C statistic.
Results
The cumulative incidence of a first major bleeding event during follow-up was 6.8% (35/515). According to the 7 scoring systems, the proportions of major bleeding ranged from 3.0% to 5.7% for low-risk, 6.7% to 9.9% for intermediate-risk, and 7.4% to 15.4% for high-risk patients. The overall predictive accuracy of the scores was poor, with the C statistic ranging from 0.54 to 0.61 and not significantly different from each other (P=.84). Only the Anticoagulation and Risk Factors in Atrial Fibrillation score performed slightly better than would be expected by chance (C statistic, 0.61; 95% confidence interval, 0.52-0.70). The performance of the scores was not statistically better than physicians' subjective risk assessments (C statistic, 0.55; P=.94).
Conclusion
The performance of 7 clinical scoring systems in predicting major bleeding events in patients receiving oral anticoagulation therapy was poor and not better than physicians' subjective assessments.
Keywords
Although the number of outpatient oral anticoagulant prescriptions in the United States increased 1.45-fold between 1998 and 2004 (from 21 to 31 million),
Anemia was defined as a hematocrit<30% in OBRI and Shireman et al;
1
the risk/benefit evaluation of oral anticoagulant therapy remains challenging. To help physicians assess the bleeding risk of their patients taking oral anticoagulant therapy, 7 clinical prediction scoring systems have been derived since 1998 (Table 1). All 7 scoring systems stratify patients into 3 categories of bleeding risk (low, intermediate, and high) and show major bleeding rates varying from 2.8% to 5.7% in the low-risk category to 7.4% to 15.4% in the high-risk category.Clinical Significance
- •Clinical scores did not accurately predict the risk of major bleeding in unselected patients receiving oral anticoagulants.
- •Clinical scores were not superior to subjective clinical judgment in predicting the risk of major bleeding.
Table 1Characteristics of the Derivation Studies and Scores
| Major Bleeds Definition | Risk Factors
(Attributed Risk Points) | Risk Categories |
| OBRI 2 | |||
| Overt bleed that led to the loss of≥2 units in≤7 d or was otherwise life-threatening | Age≥65 y (1)
Stroke (1) Gastrointestinal bleed (1) Recent myocardial infarction, anemia, diabetes or creatinine>1.5 mg/dL (1) | Low: 0
Intermediate: 1-2 High: 3-4 |
| Kuijer et al 3 | |||
| Overt bleed and a decline in hemoglobin concentration of≥20 g/L if need for transfusion of≥2 U of red blood cells, if it was retroperitoneal or intracranial, or if it warranted permanent discontinuation of treatment | Age≥60 y (1.6)
Female (1.3) Cancer (2.2) | Low: 0
Intermediate: 1-2 High: ≥3 |
| Shireman et al 4 | |||
| Hospitalizations due to a gastrointestinal bleed or an intracranial bleed according to DRG and ICD-9 CM codes | Age≥70 y (0.49)
Female (0.32) Gastrointestinal bleed>10 d (0.58) Gastrointestinal bleed<10 d (0.62) Anemia (0.86) Diabetes (0.27) Alcohol or drug abuse (0.71) Antiplatelet (0.32) | Low: ≤1.07
Intermediate: >1.07, <2.19 High: ≥2.19 |
| HEMORR2HAGES 5 | |||
| Any hospitalization for hemorrhage determined by Medicare claims | Age≥75 y (1)
Gastrointestinal bleed (2) Anemia (1) Glomerular filtration rate<30 mL/min or hepatic disease (1) Cancer (1) Hypertension (1) CYP2C9 mutation (1) Alcohol abuse (1) Low platelet (1) Fall risk (1) | Low: 0-1
Intermediate: 2-3 High: ≥4 |
| RIETE 7 | |||
| Overt bleed that required a transfusion of≥2 U of blood, or was retroperitoneal, spinal or intracranial, or was fatal | Age≥75 y (1)
Major bleed<15 d (2) Anemia (1.5) Creatinine>1.2 mg/dl (1.5) Cancer (1) Pulmonary embolism (1) | Low: 0
Intermediate: 1-4 High: ≥5 |
| HAS-BLED 8 | |||
| Any bleed requiring hospitalization or causing a decrease in hemoglobin level of>20 g/L or requiring blood transfusion that was not a hemorrhagic stroke | Age≥65 y (1)
Stroke (1) Gastrointestinal bleed (1) Creatinine≥2.3 mg/dL (1) Hepatic disease (1) Hypertension (1) Alcohol abuse (1) Drug abuse (1) Labile INR (1) | Low: 0
Intermediate: 1-2 High: ≥3 |
| ATRIA 6 | |||
| Fatal bleed, requiring transfusion of≥2 U of blood, or into a critical anatomic site (according to ICD-9 CM codes) | Anemia (3)
Glomerular filtration rate<30 mL/min (3) Age≥75 y (2) Prior bleed (1) Hypertension (1) | Low: 0-3
Intermediate: 4 High: 5-10 |
INR=international normalized ratio; OBRI=Outpatient Bleeding Risk Index; DRG=diagnosis-related group; ICD-9 CM=International Classification of Diseases, 9th Revision, Clinical Modification; HEMORR2HAGES = Hepatic or renal disease, Ethanol abuse, Malignancy, Older, Reduced platelet count or function, Rebleeding risk, Hypertension, Anemia, Genetic factors, Excessive fall risk, Stroke; RIETE=Registry of Patients with Venous Thromboembolism; HAS-BLED=Hypertension, Abnormal renal-liver function, Stroke, Bleeding history, Labile INR, Elderly, Drugs/alcohol; ATRIA=Anticoagulation and Risk Factors in Atrial Fibrillation.
4
as a hemoglobin level<13 g/L in men and <12 g/L in women in RIETE and ATRIA; and according to International Classification of Diseases, 9th Revision codes in HEMORR2HAGES.† Low platelet count or platelet dysfunction based on chart review for aspirin use or thrombocytopenia.
‡ Based on chart review for high risk of falling, dementia, Parkinson's disease, or psychiatric disease.
§ Defined as the time in therapeutic range<60%.
Most of these scoring systems, however, are limited by a retrospective derivation study design,
2
, 3
, 4
, 5
, 6
applicability to patients with a specific indication for anticoagulation only (ie, atrial fibrillation or venous thromboembolism),3
, 4
, 5
, 6
, 7
, 8
lack of external validation6
, 8
or prospective validation,3
, 4
, 7
and lack of formal comparison with subjective physician judgment.3
, 4
, 5
, 6
, 7
, 8
Thus, it is uncertain how these scores perform in a real-world setting of patients regardless of their anticoagulation indication and whether the scores are more accurate than physicians' subjective assessments. External validation of these scoring systems could help physicians choose the most accurate instrument to predict bleeding risk during oral anticoagulant therapy and to better weigh the individual risk-benefit for each patient. Our primary aim was to prospectively compare the performance of the 7 scoring systems to predict the risk of major bleeds in a real-world setting of internal medicine patients receiving an oral anticoagulant therapy, regardless of their anticoagulation indication. Our secondary aim was to compare the predictive performance of the scores with physicians' subjective assessments.Material and Methods
Study Design and Setting
This prospective cohort study was conducted in the Internal Medicine and Ambulatory Care and Community Medicine Departments of Lausanne University Hospital, Switzerland. The protocol was approved by the institutional review board, and all participants gave their informed consent.
Study Population
We included all consecutive patients aged ≥ 18 years who were receiving oral anticoagulant therapy at the time of hospital discharge or at presentation in the ambulatory clinic from January 1, 2008, to March 31, 2009. We identified patients receiving oral anticoagulation using the hospital's computerized physician order entry system for inpatients or on the basis of physician notification of study staff for outpatients. Oral anticoagulant therapy was defined as treatment with 1 of the 2 vitamin K antagonists available in Switzerland (acenocoumarol or phenprocoumon, which are comparable to warfarin
9
, 10
, 11
). We excluded patients who were unwilling or unable to give informed consent for study participation.Data Collection
A trained research nurse used standardized forms to collect patient baseline information at the time of hospital discharge or presentation to the ambulatory clinic. Data points collected included age, sex, indication for and commencement date of oral anticoagulation, international normalized ratio target levels, concomitant antiplatelet therapy, number of medications being taken, risk of falls, comorbid conditions (history of stroke, prior major bleeding, arterial hypertension, diabetes mellitus, angina pectoris, myocardial infarction, heart failure, peripheral artery disease, cancer [active or in remission < 1 year], liver cirrhosis, and alcohol or drug abuse), and laboratory values (hemoglobin, hematocrit, creatinine, platelet count, and international normalized ratio). The risk of falls was assessed by asking patients 2 validated screening questions:
12
(1) Did you fall during the last year? and if not, then (2) Did you notice any problem with gait, balance, or mobility? To assess the physicians' subjective assessments of bleeding risk, treating physicians (residents and fellows) were asked to report at patient discharge their estimated annual risk of major bleeds in their anticoagulated patients as low, intermediate, or high and on a continuous scale from 0% to 100%. Physician experience was measured as years in clinical practice. We did not collect any information on cytochrome CYP 2C9, because this mutation is only rarely measured in standard clinical practice. Likewise, we did not collect data on international normalized ratio lability because this information is only known in patients who were taking oral anticoagulants before enrollment. For calculating the scores, both variables were considered to be absent, a strategy used in the derivation of the Hypertension, Abnormal renal-liver function, Stroke, Bleeding history, Labile INR, Elderly, Drugs/alcohol (HAS-BLED) score.8
Outcome Measures
The primary outcome was the first major bleeding event within 12 months after study enrollment, defined as a fatal bleed; or a symptomatic bleed in a critical area or organ, such as intracranial, intraspinal, intraocular with threat to vision, retroperitoneal, intra-articular, pericardial, or intramuscular with compartment syndrome; or a bleed causing a decrease in hemoglobin level of ≥ 20 g/L or leading to transfusion of ≥ 2 units of whole blood or red cells.
13
, 14
A fatal bleed was defined as any death occurring within 7 days of a major bleeding episode in the absence of an alternative cause of death.15
To ascertain outcomes during follow-up, the study nurse contacted patients or caregivers via telephone at 4, 8, and 12 months after enrollment. Primary care physicians also were contacted, and hospital records were reviewed for all outcomes. We collected the following information at each follow-up interview: all international normalized ratio values, current anticoagulant and antiplatelet treatment, date and location of any bleeding, and date and cause of any death.
On the basis of the best available evidence from patient and physician interviews and medical charts, fatal and nonfatal major bleeds were adjudicated by a panel of 3 independent consultant general physicians who were blinded to the presence or absence of patient risk factors for bleeding. Final diagnoses necessitated the full consensus of this physician panel.
16
Statistical Analyses
We determined the cumulative incidence of the first major bleeding event in all patients. To compare the discriminative power of the test scores and physicians' subjective assessments, we calculated the C statistic (or area under the receiver operating characteristics curve).
17
, 18
To assess whether score performances improved when applied to the specific disease population for whom they were originally developed, we also compared the C statistic in the subgroups of patients with atrial fibrillation and venous thromboembolism as appropriate for each test. We calculated the C statistic according to the derivation studies' length of follow-up if ≤ 12 months. Although all possible effort was made to find the cause of death of all patients who died during follow-up, the cause of death in some remained unknown, and in these cases major bleeding as the cause could not be formally excluded (competing risk); we therefore also performed a sensitivity analysis using as primary outcome all deaths from any cause in addition to the first major bleeding event. A 2-sided P value < .05 was considered statistically significant. All analyses were performed using STATA 11.1 (StataCorp LP, College Station, Tex).Results
Study Population
Of 650 consecutive patients receiving oral anticoagulant therapy who were screened between January 1, 2008, and March 31, 2009, we excluded 132 patients (20.8%) because of refusal or inability to give informed consent. We further excluded 3 patients (0.5%) who withdrew consent within a few weeks of the start of the study, leaving a final sample of 515 patients. The 3 patients who withdrew consent did not have a bleeding event before withdrawal. Excluded patients were significantly older than enrolled patients (median 81 vs 71 years, P < .001) and somewhat more likely to be women (45.5% vs 36.2%, P = .06). Follow-up was complete for all 515 enrolled patients.
Baseline characteristics of our patient sample are shown in Table 2. Median age of the patients was 71.2 years (interquartile range, 61.6-79.3 years). Most patients were hospitalized at the time of enrollment (96%). The majority of the patients started anticoagulant therapy ≥ 3 months before enrollment (64.1%, n = 330). The mean duration of anticoagulation was 273 days (standard deviation, 127).
Active cancer or cancer in remission of<1 year.
Table 2Baseline Characteristics
| Characteristics | No. of Patients (n=515) | Percentage or Median (IQR 25%-75%) |
|---|---|---|
| Age | ||
| <65 y | 172 | 33.4 |
| 65-70 y | 67 | 13.1 |
| 70-74 y | 76 | 14.8 |
| ≥75 y | 200 | 38.8 |
| Men | 329 | 63.9 |
| Serum creatinine (mg/dL) | 515 | 0.98 (0.83-1.24) |
| Anemia (based on WHO criteria) | 234 | 45.4 |
| Platelet count<150 G/L | 59 | 11.4 |
| Comorbid disease: | ||
| Stroke or transitory ischemic attack | 80 | 15.5 |
| History of gastrointestinal bleed | 26 | 5.0 |
| Recent major bleed (<10 d) | 26 | 5.0 |
| Remote major bleed (>10 d) | 21 | 4.1 |
| Recent myocardial infarction (<10 d) | 39 | 7.6 |
| Remote myocardial infarction (>10 d) | 86 | 16.7 |
| Angina pectoris | 14 | 2.7 |
| Peripheral vascular disease | 61 | 11.8 |
| Diabetes mellitus | 130 | 25.2 |
| Arterial hypertension | 319 | 61.9 |
| Alcohol abuse | 43 | 8.4 |
| Drug abuse | 0 | 0 |
| Active cancer | 29 | 5.6 |
| Heart failure | 153 | 29.7 |
| Liver cirrhosis | 6 | 1.2 |
| Increased risk of falls (defined in “Materials and Methods”) | 308 | 60.0 |
| Indication for anticoagulation | ||
| Atrial fibrillation | 314 | 61.0 |
| Artificial heart valve | 43 | 8.3 |
| Aortic position | 35 | 6.8 |
| Mitral position | 12 | 2.3 |
| Deep venous thrombosis | 42 | 8.2 |
| Pulmonary embolism | 85 | 16.5 |
| Intramyocardial thrombus or cardioembolism | 8 | 1.6 |
| Other thrombus localizations | 6 | 1.2 |
| Akinetic myocardium | 35 | 6.8 |
| Peripheral or coronary artery bypass | 16 | 3.1 |
| Pulmonary arterial hypertension | 7 | 1.4 |
| Other indications | 9 | 1.7 |
| Atrial fibrillation, heart valve, or thrombosis | 448 | 87 |
| Concomitant treatment with antiplatelet drugs: | ||
| Acetylsalicylic acid only | 105 | 20.4 |
| Clopidogrel only | 11 | 2.1 |
| >1 antiplatelet drug | 45 | 8.7 |
| Nonsteroidal anti-inflammatory drug | 6 | 1.2 |
| Polypharmacy (≥4 drugs) | 438 | 85.0 |
IQR=interquartile range; WHO=World Health Organization.
† Patients could have>1 reason for oral anticoagulation therapy.
‡ Thrombus: mesenteric vein thrombosis (n=3), cervical vein thrombosis (n=1), Budd-Chiari syndrome (n=1), and septic jugular vein thrombosis (n=1).
§ Other anticoagulation indication: mitral stenosis (n=2), cardiac thermoablation (n=2), antiphospholipid syndrome (n=2), mitral valve surgery (n=1), cardiopulmonary bypass n=1, and retinal artery occlusion (n=1).
Description of Outcomes
Of the 43 major bleeding events that occurred during follow-up, 35 (81.4%) were a first event. Of the first bleeding events, 28 (80%) occurred on oral anticoagulation, 6 (17.1%) were related to a trauma or a surgical procedure, and 5 (14.3%) were fatal. The 12-month cumulative incidence of first major bleeds and fatal bleeds was 6.8% and 1.0%, respectively. The incidence of major bleeds decreased over time, with 20 events (57.1%) occurring during the first 4 months, 10 events (28.6%) occurring during months 5 to 8, and 5 events (14.3%) occurring during months 9 to 12. Over-anticoagulation, defined as an international normalized ratio greater than the upper limit of target international normalized ratio range, was present in 9 (25.7%) of the 35 first bleeding events, 8 (22.9%) were receiving concomitant antiplatelet treatment, and 3 (8.6%) were receiving antiplatelet treatment alone. The most common site of major bleeding was gastrointestinal (38%, n = 13), followed by intracerebral (17%, n = 6), urogenital (14%, n = 5), ear, nose, and throat (11%, n = 4), and miscellaneous others (20%, including postprocedural, retroperitoneal, spinal, and pulmonary). Overall, 59 patients (11.5%) died of nonbleeding causes during follow-up (including cardiorespiratory arrest [n = 14], heart failure [n = 13], respiratory failure [n = 12], cancer [n = 6], other causes [n = 7], and unknown causes [n = 7]).
Comparison of Major Bleeding and Mortality
All scoring systems classified the majority of patients (43.7%-79.6%) in the intermediate-risk category, except the Shireman et al
P for trend using Wilcoxon rank-sum trend test.
4
and Anticoagulation and Risk Factors in Atrial Fibrillation (ATRIA)6
studies, which classified most as low risk (75.2% and 58.3%, respectively) (Table 3). The proportion of major bleeds increased with increasing risk category, but the trend was not statistically significant, except for the ATRIA score (P = .01). In contrast, all-cause mortality rates increased significantly with increasing risk category for all score tests, except the HAS-BLED score.Table 3Patients' Risk of Major Bleeding and Outcomes at 12 Months (N=515)
| Risk Groups (Points) | Patients N (%) | Major Bleeds N (%) | All-Cause Mortality N (%) |
|---|---|---|---|
| OBRI 2 | |||
| Low (0) | 99 (19.2) | 3 (3.0) | 3 (3.0) |
| Intermediate (1-2) | 381 (74.0) | 29 (7.6) | 58 (15.2) |
| High (3-4) | 35 (6.8) | 3 (8.6) | 3 (8.6) |
| P=.12 | P=.03 | ||
| Kuijer et al 3 | |||
| Low (0) | 78 (15.2) | 3 (3.8) | 1 (1.3) |
| Intermediate (1.3-2.99) | 410 (79.6) | 30 (7.3) | 54 (13.2) |
| High (≥3) | 27 (5.2) | 2 (7.4) | 9 (33.3) |
| P=.33 | P=.00 | ||
| Shireman et al 4 | |||
| Low (≤1.07) | 387 (75.2) | 22 (5.7) | 37 (9.6) |
| Intermediate (1.07-2.19) | 121 (23.5) | 12 (9.9) | 27 (22.3) |
| High (≥2.19) | 7 (1.3) | 1 (14.3) | 0 (0) |
| P=.07 | P=.004 | ||
| HEMORR2HAGES 5 | |||
| Low (0-2) | 97 (18.8) | 3 (3.1) | 5 (5.2) |
| Intermediate (2-3) | 225 (43.7) | 15 (6.7) | 25 (11.1) |
| High (≥4) | 193 (37.5) | 17 (8.8) | 34 (17.6) |
| P=.07 | P=.002 | ||
| RIETE 7 | |||
| Low (0) | 138 (26.7) | 6 (4.3) | 5 (3.6) |
| Intermediate (1-4) | 351 (68.2) | 25 (7.1) | 51 (14.5) |
| High (≥4) | 26 (5.1) | 4 (15.4) | 8 (30.8) |
| P=.06 | P<.001 | ||
| HAS-BLED 8 | |||
| Low (0) | 71 (13.8) | 2 (2.8) | 4 (5.6) |
| Intermediate (1-2) | 349 (67.8) | 24 (6.9) | 48 (13.8) |
| High (≥3) | 95 (18.4) | 9 (9.5) | 12 (21.6) |
| P=.10 | P=.24 | ||
| ATRIA 6 | |||
| Low (0-3) | 303 (58.3) | 15 (5.0) | 21 (6.9) |
| Intermediate (4) | 65 (12.6) | 3 (4.6) | 7 (10.8) |
| High (5-10) | 147 (28.5) | 17 (11.6) | 36 (24.5) |
| P=.01 | P<.001 |
OBRI=Outpatient Bleeding Risk Index; HEMORR2HAGES = Hepatic or renal disease, Ethanol abuse, Malignancy, Older, Reduced platelet count or function, Rebleeding risk, Hypertension, Anemia, Genetic factors, Excessive fall risk, Stroke; RIETE=Registry of Patients with Venous Thromboembolism; HAS-BLED=Hypertension, Abnormal renal-liver function, Stroke, Bleeding history, Labile INR, Elderly, Drugs/alcohol; ATRIA=Anticoagulation and Risk Factors in Atrial Fibrillation.
Comparison of Predictive Accuracy and Discriminatory Power of Score Tests
The discriminatory power of score tests in the whole cohort, expressed as the C statistics, ranged from 0.54 to 0.61 (Table 4), with no significant differences between the scores (P = .84). No score performed better than due to chance alone except the ATRIA score (C statistic, 0.61; 95% confidence interval [CI], 0.52-0.70). By using death from any cause and the first bleeding event as a combined outcome, the discriminatory power for all scores ranged from 0.55 (95% CI, 0.50-0.60) to 0.66 (95% CI, 0.60-0.72; results not shown).
Statistical comparison across scores: P=.84.
Table 4Discriminative Power of the Scores
| Prediction Score | Current Study | Same Indication as Derivation Study † OBRI, all indications for the anticoagulation and outcomes were measured at 12 mo; Kuijer et al,3 only venous thromboembolism and outcomes were measured at 4 mo; Shireman et al,4 only atrial fibrillation and outcomes were measured at 4 mo; HEMORR2HAGES, only atrial fibrillation and outcomes were measured at 12 mo; RIETE, only venous thromboembolism and outcomes were measured at 4 mo; HAS-BLED, only atrial fibrillation and outcomes were measured at 12 mo; ATRIA, only atrial fibrillation and outcomes were measured at 12 mo. | Derivation Study |
|---|---|---|---|
| C-statistic (95% CI or SD) | |||
| OBRI 2 | 0.56 (0.50-0.62) | 0.56 (0.50-0.62) | 0.72 |
| Kuijer et al 3 | 0.54 (0.48-0.60) | 0.50 (0.28-0.73) | 0.82 (0.66-0.98) |
| Shireman et al 4 | 0.57 (0.48-0.65) | 0.59 (0.41-0.76) | 0.63 |
| HEMMOR2HAGES 5 | 0.58 (0.50-0.67) | 0.59 (0.48-0.70) | 0.67 (0.04) |
| RIETE 7 | 0.57 (0.49-0.65) | 0.28 (0.10-0.47) | NA |
| HAS-BLED 8 | 0.57 (0.49-0.65) | 0.58 (0.49-0.68) | 0.72 (0.64-0.79) |
| ATRIA 6 | 0.61 (0.52-0.70) | 0.65 (0.53-0.76) | 0.70 (0.70-0.78) |
CI=confidence interval; SD=standard deviation; OBRI=Outpatient Bleeding Risk Index; HEMMOR2HAGES = Hepatic or renal disease, Ethanol abuse, Malignancy, Older, Reduced platelet count or function, Rebleeding risk, Hypertension, Anemia, Genetic factors, Excessive fall risk, Stroke; RIETE=Registry of Patients with Venous Thromboembolism; HAS-BLED=Hypertension, Abnormal renal-liver function, Stroke, Bleeding history, Labile INR, Elderly, Drugs/alcohol; ATRIA=Anticoagulation and Risk Factors in Atrial Fibrillation; NA=not available.
† OBRI, all indications for the anticoagulation and outcomes were measured at 12 mo; Kuijer et al,
3
only venous thromboembolism and outcomes were measured at 4 mo; Shireman et al,4
only atrial fibrillation and outcomes were measured at 4 mo; HEMORR2HAGES, only atrial fibrillation and outcomes were measured at 12 mo; RIETE, only venous thromboembolism and outcomes were measured at 4 mo; HAS-BLED, only atrial fibrillation and outcomes were measured at 12 mo; ATRIA, only atrial fibrillation and outcomes were measured at 12 mo.We did not find any improvement in the performance of the scores when we restricted our analysis to patients with the specific disease (atrial fibrillation, venous thromboembolism) or follow-up period for which the scores were originally developed (Table 4).
Physicians' Subjective Assessments
The median clinical experience of the physicians was 3.0 (interquartile range, 2.0-4.5) years. The median annual risk of major bleeds based on physicians' subjective assessments was 5% (interquartile range, 2-10) when physicians were asked to estimate annual risk for each patient on a scale from 0% to 100%. Overall, 55%, 38%, and 7% of patients were estimated to have a low, intermediate, and high annual risk of bleeding, respectively. Physician estimation had a C statistic for accurately predicting major bleeds of 0.55 (95% CI, 0.46-0.65). The C statistics did not differ significantly between the predictive accuracy of the scoring systems and the physicians' subjective assessments (P = .94).
Discussion
We found that existing clinical scores poorly predict major bleeding events in unselected internal medicine patients receiving oral anticoagulant therapy. The C statistics ranged from 0.54 to 0.61 with 95% CIs crossing 0.50 for all scores except the ATRIA score. This suggests that the discriminatory power was not better than would have been expected due to chance alone for all scores except the ATRIA score, which performed only slightly better.
19
Furthermore, overall score performances were not better than physicians' subjective assessments, which had a similar C statistic of 0.55. The 12-month cumulative incidence of a first major bleeding event on oral anticoagulant therapy was 6.8%, which is comparable to previous studies.1
, 20
Our results contrast with the findings of the internal validation studies of these clinical scoring systems: In these, the C statistics for the prediction of major bleeds ranged from 0.63 to 0.82 (Table 4).
2
, 3
, 4
, 5
, 6
, 7
, 8
For example, the C statistic in the derivation study by Kuijer et al3
was 0.82 (95% CI, 0.66-0.98) compared with 0.54 (95% CI, 0.48-0.60) in our study. These differences may be attributable to several factors. First, with the exception of the Outpatient Bleeding Risk Index (OBRI), all scores were derived and internally validated in a sample of patients with specific diseases (eg, atrial fibrillation or venous thromboembolism), whereas we applied the scores to a patient population receiving oral anticoagulation for a varied range of indications. When we restricted the analyses to the clinical indications for which each score was originally intended, however, their predictive performances did not improve. Second, changes in the target anticoagulation therapeutic ranges, as well as the target population treated with oral anticoagulant therapy over these last decades, might have influenced bleeding risk. The most striking example is the OBRI, which was derived from data collected between 1977 and 1983 when more aggressive oral anticoagulant regimens were common.2
Third, patients were followed for varying lengths of time in the different studies: 3 months in the Kuijer et al,3
Shireman et al,4
and Registry of Patients with Venous Thromboembolism (RIETE) studies;7
12 months in the HAS-BLED;8
36 months in the Hepatic or renal disease, Ethanol abuse, Malignancy, Older, Reduced platelet count or function, Rebleeding risk, Hypertension, Anemia, Genetic factors, Excessive fall risk, Stroke (HEMORR2HAGES);5
48 months in the OBRI;2
and 6 years in the ATRIA study.6
Because bleeding risk is known to vary with time,21
these different follow-up times may have contributed to different bleeding risks across the studies. Although we re-ran the analyses using the same anticoagulation indications as in the derivation studies, we were able to adjust for the follow-up duration only when these were 12 months or less. Finally, the definition of major bleeds varied across studies (Table 1), which is likely to have led to differences in the observed rates of major bleeds.Our study has clinical and research implications. Existing clinical bleeding risk scores have a limited accuracy and discriminative power and do not offer a clear benefit beyond physicians' subjective assessments alone. Thus, novel clinical risk assessment models are needed that accurately and reliably predict the risk of major bleeding in patients receiving oral anticoagulant therapy.
Our study has several strengths. First, we prospectively identified consecutive patients receiving oral anticoagulation and no patient was lost to follow-up, reducing the risk of selection bias. Second, the inclusion of a broad sample of internal medicine patients increases the generalizability of our findings. Finally, all outcome events were adjudicated by a blinded, independent panel, reducing the risk of assessment bias.
Our study also has several limitations. First, we excluded approximately 21% of patients, mainly because they were unwilling or unable to give informed consent. The excluded patients were older and more likely to have been sicker and have cognitive dysfunction. Therefore, we cannot rule out the possibility that these patients would have had a higher bleeding risk than our study sample. Nevertheless, the bleeding rates in our sample were comparable to those observed in previous prospective studies of patients receiving oral anticoagulant therapy.
20
Second, we had neither information about CYP 2C9 mutations (included in the HEMORR2HAGES score) nor international normalized ratio lability (included in the HAS-BLED score) in our patient sample and assumed these to have been normal. The value of testing for the CYP 2C9 mutation in the prediction of bleeding risk is still controversial and not widely used in clinical practice.22
, 23
, 24
Third, we enrolled both naive and non-naive patients to oral anticoagulants in our study. Although the derivation studies of most of the scores included both types of patients (Shireman et al,4
HEMORR2HAGES, HAS-BLED, ATRIA), those of the Kuijer et al,3
OBRI, and RIETE studies included only oral anticoagulant-naive patients. Thus, we cannot entirely exclude the possibility that these 3 scores would have shown a better prognostic accuracy in an oral anticoagulant-naive patient sample. Finally, our follow-up duration (12 months) was shorter than in the derivation studies of OBRI, HEMORR2HAGES, and ATRIA but longer than in the derivation studies of Kuijer et al,3
Shireman et al,4
and RIETE, which may have decreased the discriminative power of these scores in our sample.Conclusions
Our results indicate that clinical scores poorly predict the risk of major bleeding in unselected patients receiving oral anticoagulant therapy. Score performances do not seem to be better than physicians' subjective assessments. Novel clinical risk assessment methods that accurately and reliably predict the risk of major bleeds among those receiving oral anticoagulant therapy are needed.
Acknowledgments
The authors thank Dr Jeffrey Schnipper, assistant professor of medicine at the Harvard School of Medicine (Boston, MA), for careful review of the article. Dr J. Schnipper played no role in the design or conduct of the study, or in the collection, management, or interpretation of the data.
References
- Bleeding complications with warfarin use: a prevalent adverse effect resulting in regulatory action.Arch Intern Med. 2007; 167: 1414-1419
- Prospective evaluation of an index for predicting the risk of major bleeding in outpatients treated with warfarin.Am J Med. 1998; 105: 91-99
- Prediction of the risk of bleeding during anticoagulant treatment for venous thromboembolism.Arch Intern Med. 1999; 159: 457-460
- Development of a contemporary bleeding risk model for elderly warfarin recipients.Chest. 2006; 130: 1390-1396
- Clinical classification schemes for predicting hemorrhage: results from the National Registry of Atrial Fibrillation (NRAF).Am Heart J. 2006; 151: 713-719
- A new risk scheme to predict warfarin-associated hemorrhage.J Am Coll Cardiol. 2011; 58: 395-401
- Predictive variables for major bleeding events in patients presenting with documented acute venous thromboembolism.Thromb Haemost. 2008; 100: 26-31
- A novel user-friendly score (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrial fibrillation: the Euro Heart Survey.Chest. 2010; 138: 1093-1100
- Warfarin or acenocoumarol: which is better in the management of oral anticoagulants?.Thromb Haemost. 1998; 80: 899-902
- [Comparison of quality and hemorrhagic risk of oral anticoagulant therapy using acenocoumarol versus warfarin].Med Clin (Barc). 2008; 131: 96-97
- Pharmacogenetic differences between warfarin, acenocoumarol and phenprocoumon.Thromb Haemost. 2008; 100: 1052-1057
- Will my patient fall?.JAMA. 2007; 297: 77-86
- Definition of major bleeding in clinical investigations of antihemostatic medicinal products in non-surgical patients.J Thromb Haemost. 2005; 3: 692-694
- Discrepancies in identification of major bleeding events in patients taking warfarin.Pharmacotherapy. 2008; 28: 1098-1103
- Fatal bleeding in patients receiving anticoagulant therapy for venous thromboembolism: findings from the RIETE registry.J Thromb Haemost. 2010; 8: 1216-1222
- Adjudication of end points in studies on substances influencing haemostasis—an example from vascular surgery.Eur J Vasc Endovasc Surg. 2008; 36: 703-704
- Accuracy assessment of prediction in patient outcomes.J Eval Clin Pract. 2008; 14: 1-10
- Application of summary receiver operating characteristics (sROC) analysis to diagnostic clinical testing.Adv Med Sci. 2007; 52: 76-82
- The meaning and use of the area under a receiver operating characteristic (ROC) curve.Radiology. 1982; 143: 29-36
- Hemorrhagic complications of anticoagulant and thrombolytic treatment: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition).Chest. 2008; 133: 257S-298S
- Risk factors for complications of chronic anticoagulation.Ann Intern Med. 1993; 118: 511-520
- Cost-effectiveness of using pharmacogenetic information in warfarin dosing for patients with nonvalvular atrial fibrillation.Ann Intern Med. 2009; 150: 73-83
- Should we be applying warfarin pharmacogenetics to clinical practice?.Ann Intern Med. 2009; 151 (W295): 270-273
- The long and winding road to warfarin pharmacogenetic testing.J Am Coll Cardiol. 2010; 55: 2813-2815
Article Info
Publication History
Published online: August 31, 2012
Footnotes
Funding: This study was supported by an intramural grant (CardioMet) from the University Hospital Lausanne , Switzerland. Dr Jacques Donzé was supported by grants provided by the Swiss National Science Foundation ( PBLAP3-131814 and PBLAP3-136815 ). The providers of the grants played no role in the design or conduct of the study, or in the collection, management, or interpretation of the data.
Conflict of Interest: None.
Authorship: All authors had access to the data and played a role in writing this manuscript.
Identification
Copyright
© 2012 Elsevier Inc. Published by Elsevier Inc. All rights reserved.
