The widespread coronavirus disease 2019 (COVID-19) has led to high morbidity and mortality. Therefore, early risk identification of critically ill patients remains crucial.

To develop predictive rules at the time of admission to identify COVID-19 patients who might require intensive care unit (ICU) care.

This retrospective study included a total of 361 patients with confirmed COVID-19 by reverse transcription-polymerase chain reaction between January 19, 2020, and March 14, 2020 in Shenzhen Third People’s Hospital. Multivariate logistic regression was applied to develop the predictive model. The performance of the predictive model was externally validated and evaluated based on a dataset involving 126 patients from the Wuhan Asia General Hospital between December 2019 and March 2020, by area under the receiver operating curve (AUROC), goodness-of-fit and the performance matrix including the sensitivity, specificity, and precision. A nomogram was also used to visualize the model.

Among the patients in the derivation and validation datasets, 38 and 9 participants (10.5% and 2.54%, respectively) developed severe COVID-19, respectively. In univariate analysis, 21 parameters such as age, sex (male), smoker, body mass index (BMI), time from onset to admission (> 5 d), asthenia, dry cough, expectoration, shortness of breath, asthenia, and Rox index < 18 (pulse oxygen saturation, SpO₂)/(FiO₂ × respiratory rate, RR) showed positive correlations with severe COVID-19. In multivariate logistic regression analysis, only six parameters including BMI [odds ratio (OR) 3.939; 95% confidence interval (CI): 1.409-11.015; P = 0.009], time from onset to admission (≥ 5 d) (OR 7.107; 95%CI: 1.449-34.849; P = 0.016), fever (OR 6.794; 95%CI: 1.401-32.951; P = 0.017), Charlson index (OR 2.917; 95%CI: 1.279-6.654; P = 0.011), PaO₂/FiO₂ ratio (OR 17.570; 95%CI: 1.117-276.383; P = 0.041), and neutrophil/lymphocyte ratio (OR 3.574; 95%CI: 1.048-12.191; P = 0.042) were found to be independent predictors of COVID-19. These factors were found to be significant risk factors for severe patients confirmed with COVID-19. The AUROC was 0.941 (95%CI: 0.901-0.981) and 0.936 (95%CI: 0.886-0.987) in both datasets. The calibration properties were good.

The proposed predictive model had great potential in severity prediction of COVID-19 in the ICU. It assisted the ICU clinicians in making timely decisions for the target population.

In December 2019, an outbreak of coronavirus disease 2019 (COVID-19) occurred in Wuhan, China[1,2]. Up to April l8, 2020, there were 83251 cases reported in China[3] and 1442312 internationally[4]. The estimated case-fatality rate associated with COVID-19 in Wuhan might be as high as 5.25% [95% confidence interval (CI): 4.98-5.51][5,6].

Most of the patients diagnosed with COVID-19 had mild disease[5,6]. For these patients, general isolation and symptomatic treatment are available, and intensive care unit (ICU) care is needed to reduce the number of deaths and alleviate the shortage of medical resources unless their condition worsened rapidly. In such a case, risk stratification of patients by severity of the illness at an early stage is necessary.

Predictive models or scoring systems have been developed to quantify the impact of risk factors. Complex statistical models were introduced during the model development step, and these could be used by clinicians for decision-making. Also, the risk of severity in patients could be easily estimated and monitored over time, thus motivating them to modify their behavior. Several scoring systems that targeted the severity prediction have already been developed. However, most of these proposed models did not include a description of the study population, and the calibration of predictions is rarely assessed[7]. Liang et al[8] developed a risk score based on the characteristics of COVID-19 patients at the time of admission to the hospital based on a nationwide cohort in China. However, this risk scoring system consisted of 10 items, which were relatively complex and time-consuming. Besides, hemoptysis occurred rarely. Moreover, this system was not compared with the other traditional acute respiratory distress syndrome (ARDS) scoring systems, and the calibration of this system has not been evaluated.

Hence, in this study, a total of 361 patients diagnosed with COVID-19 in the Shenzhen Third People’s Hospital were included and a rule-based on risk-prediction model for the severity of COVID-19 patients was developed. Moreover, the developed predicted model was also externally validated, and its performance was carefully calibrated.

With the ongoing COVID-19 pandemic and a severe shortage of ICU resources, it is critical for first-line clinicians to promptly predict the prognosis of patients with COVID-19 and optimize the undermined medical sources. As COVID-19 is a new pandemic and its clinical features are not yet fully illustrated, clinicians are more prone to estimate the severity of individual patients through their subjective experience. To solve this critical problem, our study established a risk-prediction model to estimate the prognosis of patients with severe COVID-19, in order that clinicians can more objectively calculate the severity of an individual patient and optimize the subsequent treatment. This risk-prediction model was based on the data obtained from 361 patients with severe COVID-19, in which 262 were in the derivation set, 99 in the internal validation set, and 126 in the external validation set. The model demonstrated high accuracy in predicting the severity of COVID-19 with an AUC of 0.948 when compared with other models.

Among dozens of clinical parameters affecting the prognosis of patients with COVID-19, many indices were shown to be significantly correlated with the severity of the disease. However, only six clinical and laboratory characteristics were identified as independent factors that could distinguish patients with severe COVID-19 from those with mild and moderate COVID-19. Older age (> 60 years) alone could not predict the prognosis independently, whereas comorbidity together with older age led to a worse prognosis.

The present model demonstrated that the P/F ratio was the highest risk factor for predicting the worst prognosis. Patients with lower oxygenation index at the time of admission had 16 times higher risk of developing severe COVID-19. Previously, the guidelines have recommended early identification of patients with severe pneumonia based on a lower oxygenation index[9,12]. As the clinical classification is primarily based on the oxygenation index, it is not surprising that the oxygenation index is predictive for the development of severe COVID-19. The P/F ratio showed a close relationship with the Qsp/Qt index, and a low P/F ratio indicates pulmonary parenchymal injury caused by COVID virus infection or a subsequent immunological response[13,14].

Another important finding of the present study was the benefit of timely admission after the onset of COVID-19 symptoms, which was associated with a six times higher risk of COVID-19 infection. This finding was in agreement with the China Ministry of Health Guidelines[12] which advocate early hospitalization, but different from the other practices in many other countries which did not mention early admission at all or emphasize home quarantine[15,16]. Patients with, mild and moderate COVID-19 were encouraged to stay at home to prevent the surge of COVID-19 and hospital collapse until they were critically ill[17-20]. The present study indicated that early admission could not only help avoid local transmission but also reduce the incidence of severity and criticality of COVID-19.

Patients with fever (> 37.3ºC) are more likely to progress to severe COVID-19 when compared with those without fever. Fever generally implies an ongoing immunological response to severe acute respiratory syndrome CoV-2 (SARS-CoV-2) virus. Higher body temperature might indicate a stronger response due to severe pathogenic infection or overreaction to the SARS-CoV-2 virus in some individual cases. As indicated by the research in China and Europe, the infection storm induced by cytokines and other immune molecules might cause deterioration of clinical symptoms and pulmonary damage. Unfortunately, the exact role of immunological response in the etiology of SARS-CoV-2 virus, whose pathological features are sometimes unpredictable and novel to clinicians, is currently unclear. Although SARS-CoV-2 and SARS are both caused by coronavirus, the implication of fever in patients with COVID-19 was not in concordance with that in patients with SARS. Patients who did not have fever during admission were more likely to die from SARS (44% vs 7%, P = 0.045)[21]. The physiological mechanism still remains unclear, and further studies are warranted to elucidate this mechanism. Nevertheless, the aforementioned finding is important as fever can easily be screened and assist in the early identification of severe COVID-19.

The proposed model suggested that the prognosis was partially determined by the patients’ general condition, such as the Charlson index and BMI. The Charlson index has been used to predict the long-term prognosis of patients with comorbidities. It has a specific scoring system that can also include age when predicting the lethality, and compare with other indices of comorbid disease[22]. This might give Charlson index a better predictive power, as advancing age is usually associated with chronological deterioration of physical functions. In contrast, the proposed model suggested that advanced age (> 60 years) alone was not enough to independently predict the prognosis of COVID-19, but the existing comorbidity might better predict the development of severe COVID-19 according to the data in the present study. This finding was consistent with that of previous reports which showed that comorbidity is an important independent predictor of mortality in COVID-19[6,23-25]. However, the study did not define the role of comorbidity quantitatively. Our study findings are novel in evaluating the relationship of comorbidity in patients with severe COVID-19 with Charlson index.

BMI (> 24) was associated with a 2.94 times higher risk of severe COVID-19. Higher BMI increases the risk of a variety of diseases, including cardiovascular diseases, diabetes, and other metabolism disorders, subsequently increasing the mortality rate, according to several large-cohort studies[26,27]. This was in agreement with that of a previous study, which reported on the role of obesity in predicting patients’ COVID-19 severity[28]. A retrospective study proved that obesity was a risk factor for the severity of infections caused by other respiratory viruses including influenza[29] and Middle East respiratory syndrome coronavirus[30].

According to the proposed model, D-dimer had poor predictive ability, which contradicted previous findings[25]. However, prospective studies are needed to further explore the link between abnormal D-dimer levels and the severity and mortality of patients with COVID-19[9,12].

The present study demonstrated that the NLR indicated a 2.57 times higher risk of the development of severe COVID-19. The NLR acts as a biomarker for assessing the severity of bacterial infections and the prognosis of patients with pneumonia and tumors[31,32]. An increase in the NLR indicates a poor clinical prognosis[33]. These results indicate that COVID-19 might affect a series of immune responses, including changes in T lymphocytes, serving as an important factor for predicting clinical deterioration. In addition, patients with severe illness might be associated with bacterial infection due to deteriorated immune function. The results of the present study were consistent with those of previous findings. For example, the study by Wang et al[23] followed up the laboratory test results of patients with COVID-19 and found that 5 patients had an increased neutrophil count and reduced lymphocyte count before their death[23]. Liu et al[34] suggested that patients aged ≥ 50 years and a NLR ≥ 3.13 were at high risk and should be promptly transferred to the ICU for invasive respiratory support[34].

The model with the aforementioned six predictors could be used in clinical practice to identify high-risk dialysis patients. A simple form of a probability prediction model was put forward to identify patients with severe COVID-19. The cutoff values of the probabilities that discriminated between patients with mild and severe COVID-19 are provided in Table 3. According to the Youden index, a cutoff value of (P = 0.439) yielded a sensitivity and a specificity of 0.950 and 0.259, respectively, which was optimal to identify patients with severe COVID-19 when compared with other potential models.

Several models have previously been developed for predicting the prognosis of pneumonia. CURB-65 is a relatively widely accepted model[35]. The CURB-65 score is comprised of 5 separate elements: Confusion, uremia, respiratory rate, blood pressure, and age ≥ 65 years. This was initially developed by the British Thoracic Society in 1987 as the CRB criteria[36], and was later modified to CURB-65 and validated by Lim et al[37] in an international, multicenter, derivation/validation study. This score was evaluated for predicting mortality if the case was identified as having severe pneumonia. On the contrary, CURB-65 is not exclusively designed for COVID-19 patients. Thus, some features in CURB-65 were invaluable for stratifying COVID-19. For example, confusion, urea level > 7 mmol/L, and low blood pressure were rarely observed at the time of admission, contributing little to the identification of severe COVID-19. The risk score developed by Liang et al[8] based on the characteristics of patients with COVID-19 at the time of admission consisted of 10 items, which is relatively complex and time-consuming. Besides, hemoptysis rarely occurs. Moreover, our model did not compare with the other traditional ARDS score systems, and the calibration of this system was not evaluated. Based on the clinical data, the proposed model calculated not only the factors of comorbidity information and BMI, but also special factors associated with COVID-19, such as the NLR, showing significantly improved performance.

The proposed predictive model, including the aforementioned 6 factors, assists in predicting the prognosis with high accuracy. It was promising in clinical decision-making to identify patients in the early stage with severe COVID-19. The model had high accuracy when tested using internal and external data from different places. However, the fitness of the model in other ethnicities has not yet been validated and is subjective to adjustment according to local data.

However, the present study had several limitations that undermine the prediction of the proposed prognostic model. The study included two of the largest hospitals in China, yet the model might not be generalizable to patients with COVID-19 in other hospitals. This was the largest COVID-19 study so far. Still, the findings were limited by the small sample size and unequal distribution of COVID-19 patients between the hospitals.

In summary, the proposed risk-prediction model for COVID-19 in this study showed significant association with the presence of obesity, delayed admission, fever, existing comorbidities, hypoxemia, and higher NLR. The model accurately predicted the severity of patients with COVID-19. This could practically help clinicians to perform timely interventions and control measures to prevent overcrowding or delayed diagnosis of severe COVID-19 in hospital settings. The predictive utility of the model warrants further validation and improvement. Other prospective studies can assist in developing a better model based on the present findings.