Skip to main content

Assessing mandatory stay-at-home and business closure effects on the spread of COVID- 19

Our take —

In this study of 10 countries during the spring of 2020, the authors found no evidence that countries implementing “more restrictive” NPIs such as stay-at-home orders and business closures reduced their daily growth in new COVID-19 cases more than Sweden and South Korea, who were categorized as having “less restrictive” NPIs. However, the analysis suffers from several methodological problems that undermine confidence in the results, and should not be regarded as providing much evidence in support of the conclusion that lockdown was ineffective.

Study design

Ecological

Study population and setting

This study estimated the association between “more restrictive” (vs. “less restrictive) non-pharmaceutical interventions (NPIs) and growth rates in new COVID-19 cases in 10 countries from February through the beginning of April 2020. The authors categorized 8 countries (England, France, Germany, Iran, Italy, the Netherlands, Spain and the United States) as having “more restrictive” NPIs (such as stay-at-home orders and business closures), and compared them to 2 countries (South Korea and Sweden) categorized as having “less restrictive” NPIs. Data at the subnational level (e.g., US states) were used in model estimation. Linear models were fit estimating the daily change in the natural log of confirmed COVID-19 cases; there were indicator variables for a total of 51 NPIs across countries, and fixed effects for each subnational unit and day of the week. Changes in case definitions were modeled with an indicator variable. Models were fit separately for each pairwise comparison (n=16) of countries with “more” and “less” restrictive NPIs; each policy coefficient in “more restrictive” countries was added together, and summed policy coefficients from “less restrictive” were subtracted to obtain an overall estimate of the marginal effect of imposing more restrictive NPIs.

Summary of Main Findings

The average daily growth rate in new cases across the 10 countries prior to NPI implementation was 0.32; the rate in South Korea was 0.25 and the rate in Sweden was 0.33. The estimated association between all combined NPIs on the growth rate was statistically significant in all countries, with the exception of Spain, ranging from -0.10 (95% CI: -0.06 to -0.15) in England to -0.33 (95% CI: -0.09 to -0.57) in South Korea. None of the 16 pairwise comparisons between countries with “more restrictive” NPIs and Sweden or South Korea resulted in a statistically significant association indicating a benefit for more restrictive NPIs.

Study Strengths

Data were analyzed at the subnational level.

Limitations

The timing of NPI implementation across countries was not randomly allocated; countries enacted policies in response to characteristics of the epidemic that were highly likely to have affected subsequent confirmed case counts. This endogeneity could have severely biased effect estimates and undermines confidence in the study results. Additionally, the sample size of countries was small (10), and did not include many countries for which subnational NPI and case count data are available. The comparison group of countries with “less restrictive” NPIs only included Sweden and South Korea, which may not serve as an appropriate counterfactual proxy. South Korea, for example, enacted a broad array of intensive interventions including a robust testing and contact tracing effort, which employed elements such as video surveillance, mobile phone location data, and credit card monitoring. Many important determinants of epidemic growth that vary across countries (e.g., population density, age structure, number of residents in congregate facilities, mask use, pre-intervention contact patterns) were not included and may have confounded estimates. Reliance on reported case counts is subject to ascertainment bias: testing, and growth in testing capacity, was not uniform across countries during the time period (e.g. South Korea). Additionally, no attention was paid to appropriate lags between policy implementation and outcomes; case growth rates are unlikely to have responded immediately to any policy implementation.

Value added

This study does not provide strong evidence in support of its conclusions.

This review was posted on: 12 March 2021