Skip to main content

Development and validation of the ISARIC 4C Deterioration model for adults hospitalised with COVID-19: a prospective cohort study

Our take —

This paper detailed the rigorous development and validation of the 4C Deterioration model, which included 11 routinely measured predictors of clinical deterioration (requirement for ventilation, ICU admission, or death) among patients hospitalized with suspected COVID-19 and demonstrated strong discrimination and calibration across several subgroup and sensitivity analyses. The study included almost 75,000 patients across the UK. The model had improved performance compared to existing models, but needs external validation outside of the UK. It is yet unclear how use of this model could impact clinical outcomes.

Study design

Prospective Cohort

Study population and setting

This paper detailed the development and validation of a model predicting clinical deterioration for adults hospitalized with COVID-19, using data from the ISARIC4C (International Severe Acute Respiratory and Emerging Infections Consortium Coronavirus Clinical Characteristics Consortium) study. The study included 74,944 patients (median age 75 years, 44% female, 83% white) with suspected COVID-19 across 260 hospitals in the UK, who were admitted or first assessed for COVID-19 before August 27, 2020. The primary outcome was in-hospital clinical deterioration, a composite outcome comprising ventilatory support, admission to high-dependency or intensive care unit, or death. Predictors considered were defined a priori and based on literature of routinely measured clinical variables associated with COVID-19 prognosis, and which were measured on the day of hospital admission (or first clinical suspicion of COVID-19 for cases acquired in hospital). Modeling employed logistic regression. Variables were selected with backwards elimination, continuous predictors were modeled with restricted cubic splines, internal-external validation was used to assess between-region heterogeneity and generalizability, and models were externally validated in a holdout set of the London region. Multiple imputation was used for missing data.

Summary of Main Findings

Of the 74,944 participants, 88.2% had PCR-confirmed SARS-CoV-2 infection and 43.2% (n=31,924) had an outcome of clinical deterioration (47% ventilatory support or ICU admission, 53% death). After backwards elimination in each of ten imputed datasets and across eight development regions, 11 predictors were included in the final model: age, sex, hospital-acquired infection, Glasgow coma scale score, peripheral oxygen saturation at admission, breathing room air or oxygen therapy, respiratory rate, urea concentration, C-reactive protein concentration, lymphocyte count, and presence of radiographic chest infiltrates. The pooled c-statistic from internal-external cross-validation was 0.76 (95% CI: 0.75-0.77), indicating the model distinguished well between individuals who are likely and unlikely to experience clinical deterioration. The pooled calibration intercept (-0.01, 95% CI: -0.12 to 0.09) and slope (0.99, 95% CI: 0.97 to 1.02) indicated the model was well calibrated (i.e., there was agreement between observed and predicted risks). In decision curve analysis, the final model (called the 4C deterioration model) had higher net benefit than existing models. Model performance was similar to that achieved in external validation and in subgroup analyses stratified by timing of outcome events (≤3 days vs. >3 days). Performance of the model was slightly lower (c-statistic=0.73) among the subgroup of people with hospital-acquired COVID-19, but still higher than other models.

Study Strengths

This was a very large, multi-site study with reporting according to TRIPOD guidelines, which set rigorous standards for development and reporting of prediction model studies. The model discriminated well between individuals with high and low risk of clinical deterioration and was well calibrated across several sensitivity and subgroup analyses. The model appears useful for clinical decision making, since its performance exceeds that of other existing models and out-performs even the strongest univariable predictors. Internal-external cross-validation and external validation in a holdout sample highlight the robustness of results. The model can be implemented at the time of hospitalization, and the predictors are often routinely collected.

Limitations

The model needs to be validated outside of the UK prior to international use. It is unclear how the model can improve care or outcomes, since the majority of outcomes occur shortly after hospitalization; impact studies are needed to determine whether therapeutics are more effective when targeted at individuals with the highest risk of outcome. Difficult-to-collect or rarely available (<60% of the time) variables were not considered, so models that previously included these were not evaluated. The model only applies to hospitalized patients.

Value added

The 4C model is the best-performing existing model to predict clinical deterioration.

This review was posted on: 12 March 2021