Author: Dr. Vinodh B Nanjayya
Peer Reviewers: Prof. Andrew Udy, Dr Aidan Burrell, Dr Matthew Durie
Recently, authors of an observational study, published in JAMA, changed the reported mortality for intubated coronavirus disease (COVID-19) patients from 88.1% to 24%.1,2
This blog post outlines essential points to consider while reviewing such papers which report on COVID-19 related mortality outcomes in ICU patients.
Nearly 20% of patients develop severe to critical COVID-19, and about 5-16% require ICU admission.2-4 Observational studies suggest that patients admitted to ICU have higher mortality.5 However, there is significant variation in the reported mortality for ICU patients- from 24% to > 60%.1,6,7 Such variation in reported mortality is often due to the difference in the methodology used to calculate mortality. Clinicians need to be aware of these variations while interpreting observational studies, as such mortality statistics may influence their clinical decisions.
While reviewing observational studies, readers should always follow the standards for the critical appraisal. These standards should include whether the study is a large, single centre or multicentre research and did the study include all consecutive COVID-19 patients admitted to ICU. Generally, a large multicentre, observational study which consists of all consecutive patients admitted to ICU with COVID-19 would provide precise estimates of mortality.
Apart from these factors, the most critical question to consider in the current COVID-19 pandemic context is: do all the patients included in the study have the end-point of death or discharge by the date of censoring or follow-up?
In an ideal study which looks at the mortality outcomes of patients admitted to ICU, all the patients admitted in ICU would be followed till death or discharge from the hospital. However, this is difficult in an evolving epidemic. Hence, to provide mortality data to clinicians as quickly as possible, studies published so far have set a cut-off date on the outcome (censor date) and calculated the mortality using different denominators which have created confusion in interpreting the actual mortality.
Explanation of different methods
I am going to use Figure 1 below – which shows the clinical course over time of hypothetical intubated COVID-19 patients – to explain this. There are ten patients represented by horizontal lines: the green lines represent patients discharged alive and red lines represent patients who died in the hospital. Black vertical lines at day 30 and 60 represent the censoring dates for studies after study commencement. There are only ten patients admitted at various time points between 0 and 30 days of the study.
Figure 1. Clinical course of intubated COVID patients in a theoretical study.
The study which ends its data collection, i.e. censors data, on day 30 could report the mortality statistics in two ways based on the denominator used:
Include only patients whose follow-up is complete by day 30 and then calculate the mortality. In the above example, four patients have had the end-point of either death or discharge. Therefore, the paper would report the mortality at 75%.
Include all patients who are admitted to the unit by day 30 as the denominator and calculate the mortality. In the above example, three patients out of the ten patients admitted to the unit during the study period have died. Therefore, mortality would be 30%.
The problem of the first method is that it overestimates the mortality as there are six patients still in the ICU, some of whom might survive due to ICU interventions. Studies by Arentz et al. and Zhou F et al. have followed this method to provide mortality statistics of COVID-19 patients.7,8 The weekly report of ICU outcomes of COVID-19 patients from the Intensive Care National Audit and Research Centre (ICNARC) also uses this method.9 All these studies or reports have higher mortality rates for intubated COVID-19 patients. The paper by Richardson et al. published in JAMA had initially reported a mortality of 88.1% using this method which the authors rectified to 24.5% based on the second method.1
The downside of the second method is that it mostly underestimates the mortality as it is highly unlikely that all patients who remain intubated at the time of censoring would survive their illness. Studies by Graselli et al., Bhatraju et al., and Yang et al., have used this method to report their mortality statistics.6, 10, 11
Now, let us see the outcome of patients at day 60 when the outcome of all the intubated patients admitted in ICU is known. The actual mortality of 50% is established. As mentioned earlier, none of the studies published to date have followed this approach to estimate the mortality in patients admitted to ICU.
From the above mentioned example, it is clear that one has to look at the denominator used to calculate mortality of patients admitted in ICU to know which method is being used to report mortality.
In summary, clinicians should look at the denominator used to calculate the mortality in studies evaluating the ICU outcome of COVID-19 patients. Most of the studies published so far have either overestimated or underestimated the mortality. Only large, multicentre observational studies which follow all the patients included in the study till death or discharge from hospital will be able to provide accurate mortality statistics for COVID-19 patients.
- Richardson S , Hirsch JS , Narasimhan M , et al; and the Northwell COVID-19 Research Consortium. Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area. JAMA. Published online April 22, 2020. doi:10.1001/jama.2020.6775
- Richardson S, Hirsch JS, Narasimhan M, et al; and the Northwell COVID-19 Research Consortium. Clarification of Mortality Rate and Data in Abstract, Results, and Table 2. JAMA. Published online April 24, 2020. doi:10.1001/jama.2020.7681
- WHO-China Joint Mission. Report of the WHO-China Joint Mission on Coronavirus Disease 2019 (COVID-19). Feb 28, 2020. https://www.who.int/docs/default-source/coronaviruse/who-chinajoint-mission-on-covid-19-final-report.pdf
- Guan WJ, Ni ZY, Hu Y, et al; China Medical Treatment Expert Group for Covid-19. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med. doi:10.1056/NEJMoa2002032
- Grasselli G, Pesenti A, Cecconi M. Critical care utilization for the COVID-19 outbreak in Lombardy, Italy: early experience and forecast during an emergency response. JAMA 2020; published online March 13. DOI:10.1001/jama.2020.4031.
- Grasselli G, Zangrillo A, Zanella A, et al. Baseline Characteristics and Outcomes of 1591 Patients Infected With SARS-CoV-2 Admitted to ICUs of the Lombardy Region, Italy. JAMA 2020. Published online April 6, 2020. doi: 10.1001/jama.2020.5394
- Arentz M, Yim E, Klaff L, et al. Characteristics and outcomes of 21 critically ill patients with COVID-19 in Washington State. JAMA. Published online March 19, 2020. doi:10.1001/jama.2020.4326
- Intensive Care National Audit and Research Centre Case Mix Programme Database. ICNARC COVID-19 report 2020-04-24. Available from: https://www.icnarc.org/DataServices/Attachments/Download/c5a62b13-6486-ea11-9125-00505601089b
- Zhou F, Yu T, Du R, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020;395(10229):1054-1062. doi:10.1016/S0140-6736(20)30566-3
- Bhatraju PK, Ghassemieh BJ, Nichols M, et al. Covid-19 in Critically Ill Patients in the Seattle Region — Case Series. New Eng J Med 2020. Published online March 30, 2020. doi: 10.1056/NEJMoa2004500
- Yang X, Yu Y, Xu J, et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir Med. 2020; S2213-2600(20)30079-5. Published online February 24, 2020. doi:10.1016/S2213-2600(20)30079-5