Skip to main content

Does Contact Tracing Work? Quasi-Experimental Evidence from an Excel Error in England

Our take —

This study strongly suggests that England’s contact tracing program is effective at reducing the spread of SARS-CoV-2 infections and COVID-19 mortality. It does so through using regional variation in the impact of an error that caused a delay in contact identification and notification, which the authors estimated to have resulted in a large number of cases and deaths. This study was able to use this error to robustly examine the impact of contact tracing in a way that would be difficult, if not impossible, by any other study design. Although the results could be affected by unmeasured determinants of case growth that were associated with geographical variation in delayed contact tracing, the authors carefully looked for such possible confounders.

Study design

Ecological

Study population and setting

This study is based on an analysis of regional case and mortality data across England in autumn of 2020, during which an Excel-based error caused a temporary and unexpected interruption in contact tracing efforts. The error was caused by reaching the row limit on an Excel sheet which was used to inform contact tracers about whom to contact after a positive test for SARS-CoV-2 had been discovered. As a result, an anticipated ~48,000 close contacts of 15,841 people with newly detected SARS-CoV-2 infection (15-20% of all newly reported cases during the affected period) were neither identified nor notified immediately through the contact tracing system. Although the error did not affect case notification, the error caused delays of up to one week in referral of cases to contact tracing from September 25 to October 2, 2020. The study took advantage of the fact that the error did not impact areas of England equally, and the proportion of people not processed on time differed widely across geographical regions. The geographic distribution of cases affected by the error was inferred from delays in reported cases by region. The researchers compared SARS-CoV-2 infections and COVID-19 deaths through November 1 between regions with higher and lower rates of delayed contact tracing to estimate the impact of timely contact notification on population-scale outcomes. In order to test the robustness of their work, the authors used a variety of alternative statistical models and assumptions to observe how much results changed. One key test involved matching regions with similar COVID-19 outcome trends, to attempt to eliminate the possibility that regions more affected by the error had a higher SARS-CoV-2 infection rate.

Summary of Main Findings

The study’s main finding is that the error caused a delay in contact identification and notification which in turn caused more than 125,000 SARS-CoV-2 infections and more than 1,500 COVID-19 deaths. These figures represent more than 23% of reported infections and 31% of deaths during the period up to November 1, 2020, and are at the low end of the authors’ estimates, corresponding to conservative assumptions. There were an estimated 25 additional new infections per delayed referral over the subsequent six weeks. The estimated effect on new infections was largest during the week of September 28, and waned to near zero by the week of October 26. The authors found no evidence that regions more affected by the error had any meaningful differences in determinants of new infections before the error occurred, relative to regions less affected by the error. Results were similar under several different assumptions and analytical approaches.

Study Strengths

The key strength of this study is in the potentially as-if-randomized change in the number of contacts of people infected with SARS-CoV-2 who were notified in different regions, caused by the error having different impacts across regions. That, in combination with the appropriate regional-level data, gives a highly reliable way to estimate the impact of the error, and to thus examine the impact of contact notification on the spread of COVID-19 cases and deaths. A second major strength is in the way in which the authors searched for and tested a variety of alternative explanations. These results appeared to be largely robust to a wide variety of biases, such as selection biases and confounding related to local epidemic trajectories, social factors, and economic factors.

Limitations

It is unclear what explains the regional variation in the impact of the error. If the reasons for this regional variation are related to other factors which were concurrently affecting the epidemic, it is possible that this may explain some of the results. Other key limitations in this study concern generalizability to other settings. The estimates are strongly determined by the specific regions, social settings, and health care systems in which the error occurred. A stoppage of contact tracing is likely to be very different in other regions and settings as the one that occurred in England at that specific time, as the implementation of contact tracing varies widely between countries, regions and over time. Secondly, this study was not able to directly identify individuals whose contact tracing was delayed, nor was the geographic distribution of these cases available. As a result, the authors had to infer regional variations in the amount of contact tracing delay by using lags in case reporting. While this is not inherently a major problem for the main results, it introduces some additional required assumptions, reducing its overall reliability somewhat. Finally, the delays in contact tracing may be presumed to have affected mostly extra-household contacts, and the anticipated number of such contacts per case was approximately 3. Given the large magnitude of estimated effect (25 additional infections per delayed referral), the results imply that both 1) the attack rates among extra-household contacts and 2) the probability of success in reaching contacts quickly and in their subsequent self-isolation may be implausibly high.

Value added

This study provides strong evidence of the effectiveness of government contact tracing programs on the spread of COVID-19 cases and mortality. By taking advantage of an unintended consequence of the contact tracing program’s reporting system, this study was able to estimate the impact of contact tracing reliably in a manner that would be impractical otherwise.

This review was posted on: 19 February 2021