If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Requests for reprints should be addressed to M. Todd Greene, PhD, MPH, Department of Internal Medicine, University of Michigan, 2800 Plymouth Rd., NCRC Bldg. 16, Rm. 470C, Ann Arbor, MI 48109.
The Michigan Hospital Medicine Safety Consortium Data Coordinating Center, Ann ArborDepartment of Internal Medicine, University of Michigan Medical School, Ann Arbor
The Michigan Hospital Medicine Safety Consortium Data Coordinating Center, Ann ArborDepartment of Internal Medicine, University of Michigan Medical School, Ann ArborVA Ann Arbor Health Care System, Mich
The Michigan Hospital Medicine Safety Consortium Data Coordinating Center, Ann ArborDepartment of Internal Medicine, University of Michigan Medical School, Ann Arbor
The Michigan Hospital Medicine Safety Consortium Data Coordinating Center, Ann ArborDepartment of Internal Medicine, University of Michigan Medical School, Ann ArborVA Ann Arbor Health Care System, Mich
The Michigan Hospital Medicine Safety Consortium Data Coordinating Center, Ann ArborDepartment of Internal Medicine, University of Michigan Medical School, Ann Arbor
Patients hospitalized for acute medical illness are at increased risk for venous thromboembolism. Although risk assessment is recommended and several at-admission risk assessment models have been developed, these have not been adequately derived or externally validated. Therefore, an optimal approach to evaluate venous thromboembolism risk in medical patients is not known.
Methods
We conducted an external validation study of existing venous thromboembolism risk assessment models using data collected on 63,548 hospitalized medical patients as part of the Michigan Hospital Medicine Safety (HMS) Consortium. For each patient, cumulative venous thromboembolism risk scores and risk categories were calculated. Cox regression models were used to quantify the association between venous thromboembolism events and assigned risk categories. Model discrimination was assessed using Harrell's C-index.
Results
Venous thromboembolism incidence in hospitalized medical patients is low (1%). Although existing risk assessment models demonstrate good calibration (hazard ratios for “at-risk” range 2.97-3.59), model discrimination is generally poor for all risk assessment models (C-index range 0.58-0.64).
Conclusions
The performance of several existing risk assessment models for predicting venous thromboembolism among acutely ill, hospitalized medical patients at admission is limited. Given the low venous thromboembolism incidence in this nonsurgical patient population, careful consideration of how best to utilize existing venous thromboembolism risk assessment models is necessary, and further development and validation of novel venous thromboembolism risk assessment models for this patient population may be warranted.
Several venous thromboembolism (VTE) risk assessment models (RAMs) have been developed, but lack external validation in hospitalized medical patients.
•
VTE incidence in medical patients is low (1%).
•
Results from an external validation study in a cohort of over 60,000 medical patients indicated poor model discrimination for VTE RAMs assessed.
•
Existing VTE RAMs have limited utility in identifying the highest-risk subset of medical patients for whom pharmacologic prophylaxis is warranted.
Identifying medical patients at increased risk for venous thromboembolism using an individualized approach is of increasing importance and is emphasized in available guidelines.
Prevention of VTE in nonsurgical patients: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines.
External validation of the risk assessment model of the International Medical Prevention Registry on Venous Thromboembolism (IMPROVE) for medical patients in a tertiary health system.
Multicentre validation of the Geneva Risk Score for hospitalised medical patients at risk of venous thromboembolism. Explicit ASsessment of Thromboembolic RIsk and Prophylaxis for Medical PATients in SwitzErland (ESTIMATE).
External validation of the risk assessment model of the International Medical Prevention Registry on Venous Thromboembolism (IMPROVE) for medical patients in a tertiary health system.
Additional large-scale, external validation studies are important to help confirm or refute the accuracy of available risk assessment models, especially during hospital admission and based on detailed clinical data, as the performance of existing risk assessment models has been moderate at best in this setting.
The Michigan Hospital Medicine Safety Consortium (HMS) is a state-wide quality collaborative focused on preventing adverse events in hospitalized medical patients. The consortium collects detailed patient-level data on venous thromboembolism risk factors and outcomes. The aim of the present study was to externally validate several existing risk assessment models using the large HMS cohort to determine which risk assessment model optimally predicts venous thromboembolism in acutely ill, hospitalized, medical patients.
Methods
Study Setting and Participants
The setting and design of HMS have been previously described.
Although participation is voluntary, each hospital receives payments for participating in the consortium and for data collection.
Eligible patients included those admitted to a medicine service for 2 days or longer. Patients were excluded if they met any of the following criteria: 1) under the age of 18 years; 2) pregnant; 3) underwent any surgical procedure during the admission; 4) direct admission to an intensive care unit (ICU); 5) direct admission for end-of-life care; 6) diagnosis of venous thromboembolism in the 6 months prior to admission; 7) admitted for presumed venous thromboembolism; 8) admitted under observation status; 9) re-admitted within 90 days of discharge from an admission included in the registry; or 10) received systemic anticoagulation on day 1 or day 2 of the index hospitalization.
Detailed patient demographic, medical history, predefined risk factors for venous thromboembolism, and laboratory and medication data were collected through a standardized process using a trained medical record abstractor at each hospital. Patients discharged from each participating hospital were sampled on an 8-day rolling cycle to avert bias in selecting cases for review.
Data on the first 18 eligible cases discharged during each cycle were collected. Follow-up data were collected by medical record review and direct telephone follow-up at 90 days post-hospital discharge. Each hospital is audited on an annual basis by data quality coordinators to ensure completeness and accuracy of data abstraction. The University of Michigan is the HMS coordinating center.
Ascertainment of Outcomes
The primary outcome of interest was clinically diagnosed, image-confirmed hospital-associated venous thromboembolism, including proximal upper- or proximal lower-extremity deep vein thrombosis and pulmonary embolism. In order to be considered hospital acquired, venous thromboembolism events must have occurred on the third day after admission or later during an index hospitalization. The diagnosis of deep vein thrombosis was based on positive findings via compression Doppler ultrasound or venography, whereas pulmonary embolism was confirmed via computed tomography scan, ventilation perfusion scan, or pulmonary angiography. Venous thromboembolism outcomes were assessed out to 90 days from the date of index hospital admission. Patients transferred to an ICU or palliative care, and those who died during follow-up were censored; however, venous thromboembolism events that contributed to death or were the reason for transfer to the ICU were included. Patients who were alive and free of venous thromboembolism occurrence at 90 days following admission were right-censored. Telephone follow-up at 90 days was completed for 58% of patients. Medical record review at 90 days was completed for 100% of eligible patients.
Statistical Analysis
External Validation of Existing Risk Assessment Models
All eligible patients were included as an external validation sample for existing risk assessment models. Numerous risk factors thought to increase risk for venous thromboembolism as specified by each risk assessment model were assessed for all patients. Bivariable Cox regression was used to assess the independent associations between putative risk factors and 90-day venous thromboembolism. Cumulative risk scores based on the presence of individual risk factor associated weights for the Kucher,
risk assessment models were calculated for each patient. In this manner, patients were assigned an “at-risk” status for each risk assessment model based on established cut points.
Cox regression models with gamma shared frailty by hospital were used to determine the hazards of developing venous thromboembolism for patients determined to be “at-risk” for each risk assessment model. Model discrimination was assessed via Harrell's C-index with 95% confidence intervals using the somersd package in Stata v.13 (StataCorp LP, College Station, Texas).
Calibration was assessed by comparing estimated incidence rates for both “low-risk” and “at-risk” groups for each of the existing risk assessment models. Event rates and hazard ratios for each of the represented scores across the scales of each risk assessment model were also investigated to assess calibration.
Sensitivity Analysis
Given that the pathophysiology of upper extremity deep vein thrombosis is unique from those of pulmonary embolism and lower-extremity deep vein thrombosis,
Upper vs. lower extremity deep vein thrombosis: outcome definitions of venous thromboembolism for clinical predictor rules or risk factor analyses in hospitalized patients.
we conducted sensitivity analyses for all of the validation analyses described above, only considering pulmonary embolism or lower-extremity deep vein thrombosis as the venous thromboembolism outcome definition.
Ethical and Regulatory Oversight
As the purpose of the consortium is to measure and improve the quality of existing patient care practices, this project received a “not regulated” status by the University of Michigan Medical School's Institutional Review Board.
Results
From January 2011 through March 2014, data on 63,548 eligible patients were collected from 48 Michigan hospitals. A total of 38,723 (60.9%) patients received pharmacologic venous thromboembolism prophylaxis on admission. The patients' average age was 65.8 years, and 35,264 (55.5%) were female. The average length of stay was 4.5 days (median 4 days). The distribution of select putative risk factors for venous thromboembolism is displayed in Table 1. Factors most strongly associated with venous thromboembolism risk included recent cancer diagnosis or treatment within the last year, central venous catheter present on admission, history of prior venous thromboembolism, family history of venous thromboembolism, and surgery within the 30 days prior to admission (P <.001 for all comparisons). The Figure illustrates the risk factors and respective weights used for scaling and the percentage of patients determined to be “at-risk” for each of the risk assessment models. The percentage of “at-risk” patients was <20% for existing risk assessment models assessed.
Central venous catheters were considered present on admission if patient was admitted with device or if one was inserted on day 1 or day 2 of the admission.
Immobile/not ambulating defined as having at least one of the following: immobilizing plaster cast, paralysis, or bed rest for ≥72 hours prior to hospitalization.
History of thrombophilia includes positive Factor V Leiden, positive prothrombin, G20210A gene mutation, positive lupus anticoagulant, other congenial or acquired thrombophilia.
Other acute infections include primary reason for admission based on the following Healthcare Cost and Utilization Project (HCUP) infection categories: 1.1 - Tuberculosis; 1.2 - Septicemia; 1.3 - Bacterial infection, unspecified site; 1.4 - Mycoses; 1.5 - HIV infection; 1.6 - Hepatitis; 1.7 - Viral infections; 1.8 - Other infections, including parasitic; 1.9 - Sexually transmitted infection (not HIV or hepatitis); 6.78 - Other CNS infection and poliomyelitis; 6.9 - Inflammation, infection of the eye; 8.126 - Other upper respiratory infection; 9.135 - Intestinal infection; 10.159 - Urinary tract infections; 12.197 - Skin and subcutaneous tissue infection; and 13.201 - Infective arthritis and osteomyelitis.
8777 (13.81)
1.23 (1.00-1.51)
.05
Congestive heart failure
5928 (9.33)
1.13 (0.87-1.45)
.36
Sepsis (<1 mo)
6555 (10.32)
1.10 (0.86-1.40)
.45
Obesity (BMI >30)
22,369 (35.20)
0.82 (0.69-0.96)
.02
Myocardial infarction (<1 mo)
1061 (1.67)
0.86 (0.44-1.65)
.65
Inflammatory bowel disease
2015 (3.17)
0.78 (0.48-1.26)
.31
Stroke
3037 (4.78)
0.40 (0.23-0.69)
.001
BMI = body mass index; CI = confidence interval; HR = hazard ratio.
∗ Cancer diagnosis or acute cancer treatment within the last year.
† Central venous catheters were considered present on admission if patient was admitted with device or if one was inserted on day 1 or day 2 of the admission.
‡ Immobile/not ambulating defined as having at least one of the following: immobilizing plaster cast, paralysis, or bed rest for ≥72 hours prior to hospitalization.
§ History of thrombophilia includes positive Factor V Leiden, positive prothrombin, G20210A gene mutation, positive lupus anticoagulant, other congenial or acquired thrombophilia.
‖ Other acute infections include primary reason for admission based on the following Healthcare Cost and Utilization Project (HCUP) infection categories: 1.1 - Tuberculosis; 1.2 - Septicemia; 1.3 - Bacterial infection, unspecified site; 1.4 - Mycoses; 1.5 - HIV infection; 1.6 - Hepatitis; 1.7 - Viral infections; 1.8 - Other infections, including parasitic; 1.9 - Sexually transmitted infection (not HIV or hepatitis); 6.78 - Other CNS infection and poliomyelitis; 6.9 - Inflammation, infection of the eye; 8.126 - Other upper respiratory infection; 9.135 - Intestinal infection; 10.159 - Urinary tract infections; 12.197 - Skin and subcutaneous tissue infection; and 13.201 - Infective arthritis and osteomyelitis.
FigureRisk assessment model description. BMI = body mass index; CHF = congestive heart failure; MI = myocardial infarction; PICC = peripherally inserted central catheter; RAM = risk assessment model; VTE = venous thromboembolism. aHistory of thrombophilia includes positive Factor V Leiden, positive prothrombin, G20210A gene mutation, positive lupus anticoagulant, other congenial or acquired thrombophilia. bCurrent cancer defined as cancer diagnosed or treated within the year prior to hospital admission. cImmobile defined as having at least one of the following: immobilizing plaster cast, paralysis, or bed rest for ≥72 hours prior to hospitalization. dOther acute infections include primary reason for admission based on the following Healthcare Cost and Utilization Project (HCUP) infection categories: 1.1 - Tuberculosis, 1.2 - Septicemia, 1.3 - Bacterial infection, unspecified site, 1.4 - Mycoses, 1.5 - HIV infection, 1.6 - Hepatitis, 1.7 - Viral infections, 1.8 - Other infections, including parasitic, 1.9 - Sexually transmitted infection (not HIV or hepatitis), 6.78 - Other CNS infection and poliomyelitis, 6.9 - Inflammation, infection of the eye, 8.126 - Other upper respiratory infection, 9.135 - Intestinal infection, 10.159 - Urinary tract infections, 12.197 - Skin and subcutaneous tissue infection, and 13.201 - Infective arthritis and osteomyelitis. ePICC presence defined as admitted with PICC or inserted on day 1 or 2 of hospitalization.
Among the eligible patients, 1183 (1.86%) were transferred to intensive care for reasons other than venous thromboembolism or transitioned to palliative/comfort care. Additionally, 3968 (6.24%) died between the index hospitalization admission and 90 days. Venous thromboembolism was associated with 3 transfers to the ICU and 10 deaths. A total of 670 (1.05%) patients developed a venous thromboembolism during follow-up. Two-by-two tables displaying the number of venous thromboembolism events and receipt of pharmacologic prophylaxis by “at-risk” status for each risk assessment model are presented in Table 2.
Table 290-day Venous Thromboembolism and Receipt of Pharmacologic Prophylaxis by Risk Strata per Risk Assessment Model
Overall, 5,385,199 patient-days of follow-up were observed. The hazards of venous thromboembolism and model performance statistics estimated for each of the existing risk assessment models, by risk level and receipt of pharmacologic prophylaxis, are shown in Table 3. The hazards of 90-day venous thromboembolism were approximately threefold greater in the at-risk group relative to the low-risk group for all risk assessment models. Venous thromboembolism incidence in the low-risk group as defined by the Intermountain risk assessment model was the lowest of all risk assessment models assessed. Model discrimination was poor to moderate for all risk assessment models. The Intermountain risk assessment model also demonstrated the highest (albeit, moderate) degree of discrimination (Harrell's C-index = 0.61 [95% confidence interval, 0.60-0.62]). Receipt of pharmacologic prophylaxis during the index hospitalization was not found to be an independent predictor of 90-day venous thromboembolism, and model performance was essentially unchanged by receipt of pharmacologic prophylaxis for all risk assessment models assessed (Table 3). Additional calibration results based on the distributions of the 90-day venous thromboembolism event rates and hazard ratios for each score level per risk assessment model are illustrated in Supplementary Table 1 (available online).
Table 390-day Venous Thromboembolism Incidence and Binary Risk Score Analysis by Risk Assessment Model
Numerators in each “at-risk incidence” and “low-risk incidence” cells reflect number of VTEs and denominators within these cells represent total number of patient days. Incidence rates presented in parentheses per 10,000 patient-days.
Hazard ratios and discrimination metrics based on binary risk models with “at-risk” per the respective scales of each risk assessment model coded as 1. As such, hazard ratios above 1 reflect increased hazard of VTE for patients in the respective “at-risk” categories. Harrell's C-index generated in somersd package in Stata.
Receipt of pharmacologic VTE prophylaxis was defined as receipt of any of the following treatments on day 1 or 2 of the index hospitalization: heparin 5000 units twice a day (BID); heparin 5000 units three times a day (TID); heparin 7500 units TID (for morbid obesity); enoxaparin 40 mg daily; enoxaparin 30 mg daily (for creatinine clearance <30 mL/min); enoxaparin 30 mg BID; dalteparin 5000 units daily; or fondaparinux 2.5 mg daily.
during index hospitalization (n = 38,723)
At-risk incidence
98/302,398 (3.24)
148/513,909 (2.88)
120/342,432 (3.50)
170/577,667 (2.94)
Low-risk incidence
301/2,980,961 (1.01)
251/2,769,450 (0.91)
279/2,940,927 (0.95)
229/2,705,692 (0.85)
HR (95% CI)
3.08 (2.38-3.99)
2.99 (2.37-3.76)
3.71 (2.91-4.72)
3.19 (2.54-4.00)
Harrell's C (95% CI)
0.551 (0.545-0.558)
0.590 (0.582-0.597)
0.560 (0.553-0.567)
0.601 (0.593-0.609)
No pharmacologic prophylaxis during index hospitalization (n = 24,825)
At-risk incidence
70/215,899 (3.24)
99/318,173 (3.11)
79/244,875 (3.23)
130/380,651 (3.42)
Low-risk incidence
201/1,885,941 (1.07)
172/1,783,667 (0.96)
192/1,856,965 (1.03)
141/1,721,189 (0.82)
HR (95% CI)
3.66 (2.70-4.97)
3.61 (2.71-4.80)
3.85 (2.87-5.17)
4.16 (3.14-5.50)
Harrell's C (95% CI)
0.581 (0.572-0.590)
0.616 (0.606-0.626)
0.588 (0.578-0.597)
0.625 (0.614-0.635)
CI = confidence interval; HR = hazard ratio.
∗ Numerators in each “at-risk incidence” and “low-risk incidence” cells reflect number of VTEs and denominators within these cells represent total number of patient days. Incidence rates presented in parentheses per 10,000 patient-days.
† Hazard ratios and discrimination metrics based on binary risk models with “at-risk” per the respective scales of each risk assessment model coded as 1. As such, hazard ratios above 1 reflect increased hazard of VTE for patients in the respective “at-risk” categories. Harrell's C-index generated in somersd package in Stata.
‡ Receipt of pharmacologic VTE prophylaxis was defined as receipt of any of the following treatments on day 1 or 2 of the index hospitalization: heparin 5000 units twice a day (BID); heparin 5000 units three times a day (TID); heparin 7500 units TID (for morbid obesity); enoxaparin 40 mg daily; enoxaparin 30 mg daily (for creatinine clearance <30 mL/min); enoxaparin 30 mg BID; dalteparin 5000 units daily; or fondaparinux 2.5 mg daily.
After modifying our venous thromboembolism outcome definition to include only pulmonary embolism or lower-extremity deep vein thromboses (or both), the 63,548 patients contributed a total of 5,391,673 patient-days of follow-up. A total of 512 (0.81%) patients had a 90-day venous thromboembolism using this modified outcome definition. Overall, risk assessment model performances for the modified venous thromboembolism definition were similar to those identified in our primary analyses (see Supplementary Table 2, Supplementary Table 3; available online).
Discussion
Several important findings emerged from our risk assessment model derivation and validation study of 90-day venous thromboembolism outcomes in a multi-site, large cohort of acutely ill hospitalized medical patients. First, overall venous thromboembolism incidence in this population was low (∼1%). Second, existing risk assessment models to be used at hospital admission classified <20% of our patient population as being “at-risk” for venous thromboembolism, suggesting either that: 1) risk stratification algorithms of existing risk assessment models may not adequately identify at-risk subsets of nonsurgical, hospitalized medical patients, or 2) using these risk-stratification methods the majority of this patient population is at low risk for venous thromboembolism. Third, although existing binary risk assessment models demonstrated good calibration, discrimination characteristics were uniformly poor to moderate across models. Taken together, these findings suggest that models for at-admission risk assessment in hospitalized medical patients are suboptimal and are deserving of further enquiry.
The preexisting risk assessment models assessed in our external validation study have all been evaluated in medical populations. Many criticisms of the existing risk assessment models–derivation populations comprised mostly of surgical or cancer patients, risk factor selection based on expert consensus, and lack of external validation–have been noted.
devised a computer-alert program to remind physicians to prescribe venous thromboembolism prophylaxis for high-risk patients. They developed their risk assessment model in a cohort of medical and surgical patients, with cancer present in 80% of their population. In our external validation analyses, the Kucher risk assessment model yielded the poorest discrimination characteristics, raising questions as to its generalizability to all medical patients.
developed the Padua risk assessment model, using a cohort of patients admitted to an internal medicine service in Italy. Limitations of this study include its single-center setting, non-evidence-derived data assessments, small number of venous thromboembolism events, no presentation of model performance characteristics, and suboptimal external validation.
Multicentre validation of the Geneva Risk Score for hospitalised medical patients at risk of venous thromboembolism. Explicit ASsessment of Thromboembolic RIsk and Prophylaxis for Medical PATients in SwitzErland (ESTIMATE).
compared with 17% in our validation cohort. In addition to a smaller percentage of cancer patients in our validation cohort, our operationalization of immobility classified only 5% of patients as immobile, compared with 23% in the original derivation cohort.
The IMPROVE multinational registry was designed to examine venous thromboembolism prophylaxis practices and clinical outcomes in hospitalized medical patients. The initial derivation of the predictive IMPROVE risk assessment model included 4 risk factors available at hospital admission (history of previous venous thromboembolism, known thrombophilia, cancer, and age >60 years) that were independently associated with venous thromboembolism events and yielded a c-statistic of 0.65.
Applying the predictive IMPROVE risk assessment model in our validation cohort classified approximately 12% of patients as at risk and demonstrated poorer discrimination characteristics compared with the original derivation cohort. Due to differences in clinical characteristics and uncertainty in the need for pharmacologic prophylaxis in upper- vs lower-extremity deep vein thrombosis,
Upper vs. lower extremity deep vein thrombosis: outcome definitions of venous thromboembolism for clinical predictor rules or risk factor analyses in hospitalized patients.
the risk assessment model developed in the original derivation study excluded upper-extremity deep vein thrombosis events from the combined venous thromboembolism outcome. In our sensitivity analysis excluding upper-extremity deep vein thrombosis, we found that the predictive IMPROVE risk assessment model did yield marginally better calibration, but discrimination characteristics were comparable with those detected in our primary analyses. Given that a patient's risk status may change during the course of hospitalization, and especially to determine a patient's continued venous thromboembolism risk after hospital discharge, the original IMPROVE study also developed an associative model, including the additional risk factors of immobility, lower limb paralysis, and ICU stay, which yielded an improved c-statistic of 0.69.
Recent large-scale external validation studies of the associative IMPROVE risk assessment model have shown good calibration and discrimination, suggesting that the IMPROVE associative venous thromboembolism risk assessment model may reliably stratify venous thromboembolism risk,
External validation of the risk assessment model of the International Medical Prevention Registry on Venous Thromboembolism (IMPROVE) for medical patients in a tertiary health system.
and this model has been incorporated into a recent large multicenter, multinational trial of post-hospital-discharge thromboprophylaxis, the MARINER trial (ClinicalTrials.gov Identifier: NCT02111564). However, the main limitation of the IMPROVE associative risk assessment model is incomplete validation during hospital admission. We were unable to externally validate this associative model in our study, primarily due to data censoring when a patient is transferred to the ICU in our study cohort.
derived the Intermountain risk assessment model using administrative and electronic medical record data from 143,000 medical patient admissions. Four risk factors (history of previous venous thromboembolism, an order for bed rest, peripherally inserted central venous catheter, and cancer) were predictive of hospital-associated venous thromboembolism in up to 90 days of follow-up, with good discrimination in their derivation and validation cohorts (c-statistics = 0.87 and 0.84, respectively).
Although the discrimination of the Intermountain risk assessment model in our validation cohort was the best of the risk assessment models assessed, underlying patient differences between the Intermountain cohorts and our validation cohort likely account for poorer performance in our validation sample. The prevalence of each of the 4 risk factors included in this risk assessment model was greater among the original derivation cohort compared with our validation cohort. Most notably, approximately 45% of patients in the Intermountain derivation cohort had a history of cancer.
Our study has limitations. First, obtaining individual patient-level venous thromboembolism risk factors was constrained by each abstractor's ability to retrospectively find information in patient charts. However, data collection was standardized and quality ensured through use of annual audits at each hospital. Although we employed a reliable data-collection strategy and made every effort to match how venous thromboembolism risk factors were operationalized, differences in how we implemented venous thromboembolism risk factors (eg, immobility) relative to the derivation studies may exist. Second, although complete 90-day follow-up chart review was completed for 100% of the patients, the successful telephone follow-up rate to identify venous thromboembolism events without medical record documentation was 58%. Therefore, it is possible that some venous thromboembolism events were missed after hospital discharge, especially if these were treated at another institution. However, our reported venous thromboembolism incidence in medical patients is comparable with other estimates in the literature.
Venous thromboembolism prophylaxis in hospitalized medical patients and those with stroke: a background review for an American College of Physicians Clinical Practice Guideline.
Third, given that risk assessment strategies in medical patients often follow a binary approach (at-risk vs low-risk), we assessed model discrimination based upon binary risk prediction models for each risk assessment model. These performance metrics may not align with those reported in derivation studies that report (explicitly or otherwise) c-statistics based on the full regression models containing all respective risk factors as independent predictors.
Despite these limitations, our study has several strengths. First, our study provides external validation of several existing risk assessment models within a large, representative, multi-site, cohort of hospitalized medical patients–a population that comprises the greatest proportion of patients in hospitals. Second, our standardized data collection by trained medical record abstractors coupled with centralized audits ensures data reliability and avoids problems often associated with retrospective datasets (eg, missing data, misclassification of International Classification of Diseases, 9th Revision coding) to determine venous thromboembolism risk factors and outcomes. Third, to ensure transparency of our results, we followed the recently published TRIPOD checklist for presenting results on risk prediction model validation studies.
In doing so, we have shown that several existing risk assessment models have limited ability in identifying the greatest at-risk population for whom pharmacologic prophylaxis may benefit. Given the low incidence of venous thromboembolism in hospitalized medical (non-ICU, nonsurgical) patients, the risk assessment models described in this work may have greater utility in identifying subsets of this particular patient population for whom pharmacologic prophylaxis is not warranted.
Acknowledgment
The authors thank Jason Mann, MSA for his assistance with proofing and submitting this manuscript.
Supplementary Data
Supplementary Table 190-day Venous Thromboembolism Events and Hazard Ratios at Each Risk Level per Risk Assessment Model
Supplementary Table 2Incidence and Binary Risk Score Analysis for 90-day Pulmonary Embolism or Lower-Extremity Deep Vein Thrombosis per Risk Assessment Model
Kucher
Padua
IMPROVE
Intermountain
VTE events/patient-days (incidence per 10,000 patient-days)
Supplementary Table 390-day Pulmonary Embolism or Lower-Extremity Deep Vein Thrombosis Events and Hazard Ratios at Each Risk Level per Risk Assessment Model
Score
VTE
No VTE
Event Rate
HR (95% CI)
Kucher
0
66
15,756
0.42%
ref
1
187
29,052
0.64%
1.58 (1.19-2.09)
2
53
8273
0.64%
1.60 (1.11-2.29)
3
69
3519
1.92%
5.03 (3.59-7.05)
4
88
4357
1.98%
5.27 (3.83-7.25)
5
26
1308
1.95%
5.08 (3.23-8.00)
6
12
397
2.93%
7.89 (4.27-14.60)
7
7
272
2.51%
7.02 (3.22-15.30)
8
2
80
2.44%
7.31 (1.79-29.87)
9
2
16
11.11%
38.25 (9.36-156.31)
10
0
6
0.00%
-
Padua
0
41
10,712
0.38%
ref
1
118
20,340
0.58%
1.54 (1.08-2.20)
2
96
14,946
0.64%
1.73 (1.20-2.50)
3
64
6645
0.95%
2.66 (1.80-3.94)
4
90
4876
1.81%
5.23 (3.61-7.57)
5
52
3221
1.59%
4.64 (3.08-6.99)
6
25
1239
1.98%
5.96 (3.62-9.80)
7
16
571
2.73%
8.14 (4.57-14.52)
8
7
301
2.27%
7.09 (3.18-15.81)
9
2
124
1.59%
5.06 (1.22-20.93)
10
0
43
0.00%
-
11
1
15
6.25%
26.92 (3.70-195.94)
12
0
2
0.00%
-
13
0
1
0.00%
-
IMPROVE
0
67
20,298
0.33%
ref
1
280
35,461
0.78%
2.50 (1.91-3.26)
2
71
3163
2.20%
8.07 (5.77-11.27)
3
27
1027
2.56%
7.97 (5.10-12.46)
4
56
2732
2.01%
6.47 (4.54-9.22)
5
10
267
3.61%
13.96 (7.18-27.15)
6
1
44
2.22%
6.82 (0.95-49.11)
7
0
40
0.00%
-
8
0
4
0.00%
-
Intermountain
0
286
51,108
0.56%
ref
1
192
10,731
1.76%
3.39 (2.83-4.08)
2
33
1142
2.81%
5.74 (4.00-8.23)
3
1
53
1.85%
4.56 (0.64-32.50)
4
0
2
0.00%
-
CI = confidence interval; DVT = deep vein thrombosis; HR = hazard ratio; VTE = venous thromboembolism.
Prevention of VTE in nonsurgical patients: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines.
External validation of the risk assessment model of the International Medical Prevention Registry on Venous Thromboembolism (IMPROVE) for medical patients in a tertiary health system.
Multicentre validation of the Geneva Risk Score for hospitalised medical patients at risk of venous thromboembolism. Explicit ASsessment of Thromboembolic RIsk and Prophylaxis for Medical PATients in SwitzErland (ESTIMATE).
Upper vs. lower extremity deep vein thrombosis: outcome definitions of venous thromboembolism for clinical predictor rules or risk factor analyses in hospitalized patients.
Venous thromboembolism prophylaxis in hospitalized medical patients and those with stroke: a background review for an American College of Physicians Clinical Practice Guideline.
The general overview and preliminary results of this manuscript have been previously presented at the Society of Hospital Medicine Annual Meeting, Las Vegas, Nev, March 26, 2014.
Funding: Support for HMS is provided by Blue Cross and Blue Shield of Michigan and Blue Care Network as part of the BCBSM Value Partnerships program. Although Blue Cross Blue Shield of Michigan and HMS work collaboratively, the opinions, beliefs and viewpoints expressed by the authors do not necessarily reflect the opinions, beliefs and viewpoints of BCBSM or any of its employees.
Conflict of Interest: MTG and VC have no conflicts of interest to disclose. PJG reports receiving royalties from Wiley Publishing and compensation for expert witness testimony. SK reports receiving compensation for consultancies for Boehringer Ingelheim, Daiichi Sankyo, Janssen, CSL Behring, and Bristol-Myers Squibb. SJB reports receiving compensation as a consultant for Blue Care Network and the Michigan Department of Community Health, honoraria for various talks, and grants from Blue Cross Blue Shield of Michigan, Michigan Health and Hospital Association, Department of Veterans Affairs, National Institutes of Health, the Agency for Healthcare Research and Quality, and compensation for expert testimony. ACS reports compensation as a consultant for Boehringer Ingelheim, Jansen, Bayer, Daichi-Sankyo, BMS, Pfizer, and as a member of the ATLAS antithrombotic clinical trials academic research organization. SAF reports receiving compensation for consultancies for the Institute for Healthcare Improvement and the Society of Hospital Medicine; royalties from Wiley Publishing; honoraria for various talks at hospitals as a visiting professor; grants from the Centers for Disease Control and Prevention Foundation, Blue Cross Blue Shield of Michigan, Michigan Health and Hospital Association, and the Agency for Healthcare Research and Quality; and compensation for expert witness testimony. Role of the Funder/Sponsor: Blue Cross Blue Shield of Michigan and Blue Care Network supported data collection at each participating site and funded the data-coordinating center but had no role in study concept, interpretation of findings, or preparation, review, or final approval of the manuscript.
Authorship: MTG takes responsibility for the content of the manuscript, including the data and analysis; MTG had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. MTG, VC, PJG, SK, SJB, ACS, and SAF contributed substantially to the study design, data interpretation, and the writing of the manuscript.
30359-X&id=gr1.jpg){kind=link}