Retrospective Cohort, Other
Study population and setting
This study externally validated 22 prognostic models aiming to predict clinical deterioration or death among COVID-19 patients with an independent dataset of 411 patients admitted to the University College Hospital London from February 1 to April 30, 2020. The 22 prognostic models were selected from a rigorously conducted systematic review of COVID-19 prognostic models published through May 5, 2020. Participants were included in the external validation if they had PCR-confirmed SARS-CoV-2 or clinically diagnosed COVID-19 within 5 days of hospital admission; patients transferred from other hospitals were excluded.
Summary of Main Findings
Among the 411 participants in the external validation dataset (median age 66 years, 61% male, 90% PCR-confirmed), 180 had an outcome of clinical deterioration (death or initiation of ventilation support other than standard oxygen therapy), of whom 115 died. The best performing models were the NEWS2 score for predicting deterioration within 24 hours and the final model by Carr and colleagues that predicted deterioration over 14 days, both of which achieved an area under the receiver operating characteristic curve (AUROC) of 0.78 and appeared relatively well calibrated (agreement between observed and predicted outcome risk). Admission oxygen saturation and age were the strongest univariate predictors of clinical deterioration and mortality, respectively, with AUCs of 0.76. In decision curve analyses, none of the prognostic models had greater net benefit than oxygen saturation for predicting clinical deterioration or age for predicting mortality.
Missing data were multiply imputed. Results were consistent across separate sensitivity analyses that excluded patients without PCR-confirmed SARS-CoV-2 infection, excluded patients clinically deteriorated within 4 hours of hospital admission, and excluded patients with any missing data.
This was a single-site validation, and it is not clear how comparable this study population is to either the study populations from the original studies or their intended target populations; multi-site validation studies are needed. The study only included prognostic models developed during the early phase of the pandemic (prior to May 5, 2020), and almost half were developed with datasets from China (n=10), so the degree to which these results apply to current COVID-19 outcomes or study populations is unclear. There was substantial missing data for lactate dehydrogenase and D-dimer, and though it was multiply imputed, future studies with more complete data will help to identify the importance of these predictors. Several models could not be replicated with the external validation dataset due unavailability of certain data (e.g., imaging).
This is the first systematic external validation of multiple existing prognostic models for COVID-19 outcomes using an independent dataset.
This review was posted on: 10 November 2020