Study population and setting
This study used the implementation dates of a large variety of non-pharmaceutical policy interventions and assed their relationship with COVID-19 time varying reproductive number, Rt, defined as the mean number of secondary cases that one index case will infect at time t) across 130 countries and territories between January 1 and June 22, 2020. Data on non-pharmaceutical COVID-19 policies, categories, implementation dates, and a general index of strength of COVID-19 policy response were obtained from the Oxford COVID-19 Government Response Tracker. Estimates of the Rt across regions and time were from EpiForecasts.
There were four primary analyses. The first main analysis used a statistical regression model to characterize how different types of NPIs were rolled out over time, such as when they occurred and in what order. The second model observed how those NPIs were clustered in time (i.e. what policies were more likely to occur together with what other policies, forming a group or a cluster of policies). A third model attempted to determine the time lags between policy implementation and their effect on Rt by assessing 3 different time lags (1, 5, and 10 days) and estimating goodness-of-fit statistics for each model. Finally, the study used panel regression models to attempt to disentangle the impact of different types of NPIs on Rt accounting for different time lags and the effort at which NPI was implemented (any effort and maximum effort).
Summary of Main Findings
First, the study finds a major increase in NPI intensity across the world in mid-March, followed by a slow reduction in the stringency of interventions. Second, the study finds substantial evidence that NPI policies were more likely to be rolled out in particular orders, typically clustered together in time, depending in part on how policy intensity is defined (any effort or maximum effort). Thirdly, the study found support for lag times between policy implementation and Rt impact between 1-10 days. Finally, the study finds evidence that school closure and internal movement restrictions, and high-intensity public events cancellations and restrictions on gatherings. There was some evidence for impact for workplace closure, income support and debt/contract relief. Evidence for impact was inconclusive for stay-at-home requirements, public information campaigns, public transport closure, international travel controls, testing, and contact tracing.
This study critically examines and demonstrates how NPI policies are related to each other, and assesses lags in their impact. These are issues that are critical both to this analysis, and many other policy analyses which often do not acknowledge that NPI policies are highly correlated with respect to timing of implementation, and that interventions have lagging effects, due to changing compliance over time. The study uses well-vetted data appropriate for policy impact evaluation. The discussion section contains a frank and well-written interpretation of the results and the interpretable limitations thereof. We find that the findings that NPIs overall had substantial impact on Rt to be relatively robust.
While the discussion section involves cautious interpretation of impact of individual NPI because they were implemented at similar times, the highest impact section (the abstract) does not heed that caution, and strongly implies that they identified which specific types of NPIs were most effective despite temporal correlation. The key difficulty – one examined and partially established in the paper itself – is that these interventions are temporally related to each other, and also have time lagged effects in similar timescales as the policy rollouts. One major issue is that there appears to be substantial limitations in the way in which lagged effects were measured. Lags were assumed to be at a fixed amount of time and the same for all policies, but in reality policy lag effects are spread out over a wide period of time, and will be different for different policies in different situations. This, in addition to the close clustering in time between policies, makes it difficult to conclude which NPIs had the strongest evidence for impact.
The study shows strong evidence that NPI policies are related to each other over time and how that creates difficulty in examining their impact.
This review was posted on: 20 October 2020