Skip to main content

Our take —

While the puzzle is far from solved, this study, available as a preprint and thus not yet peer reviewed, provides evidence that the novel SARS-CoV-2 variant of concern 202012/01 is associated with substantial increased transmissibility. No evidence was found to suggest that the new variant was associated with increased hospitalization or death rates. The study also reported that additional stricter mitigation measures along with high rates of vaccination will be necessary to control transmission due to the estimated increased transmissibility of the new variant and its increasing frequency in the population. Testing and sampling biases may partially explain the observed results, and founder and super spreading events cannot be fully ruled out as explanations for increasing frequency and transmissibility of the new variant.

Study design

Ecological, Modeling/Stimulation

Study population and setting

This study reports on the frequency and transmissibility of SARS-CoV-2 variant of concern (VOC) 202012/01 in England between September 28 and December 1, 2020. The frequency and proportion of the VOC in England was estimated from routine diagnostic PCR data. Diagnostic assays fail to detect the spike glycoprotein (S-gene) in the VOC, and so samples exhibiting S-gene target failure (i.e. S gene drop-out) that are positive for other SARS-CoV-2 genes, can be used as a proxy for VOC in the absence of whole-genome sequencing, and was shown to be 97% specific for the VOC in the UK during the analysis period. After estimating the proportion of all positive samples with S gene drop-out, the authors used a two-strain age and regionally structured transmission model fit to COVID-19 hospitalization and death data from seven regions to explain observed increases in COVID-19 diagnoses and frequency of the VOC during the observation period. Four hypotheses to explain the coincident rise in overall cases and the VOC were evaluated: 1) increased infectiousness of the VOC; 2) immune escape (i.e., reinfection of individuals who had been previously infected with a different variant by the VOC); 3) increased susceptibility to the VOC among children as compared to other variants, and 4) a shorter generation time. The model was also used to assess whether there were differences in odds of hospitalization or relative risk of death due to the VOC. The authors also assessed whether changes in contact patterns over time might explain observed changes in transmission using Google mobility data and age-specific contact data from the CoMix social contact survey. Lastly, the authors projected the course of the epidemic under various control strategies, including lockdowns, school closures, and mass vaccination, through the summer.

Summary of Main Findings

The authors found that the proportion of COVID-19 cases in England due to the VOC has rapidly increased since November. The increasing proportion of VOC cases was also associated with a significant increase in the effective reproductive number, Rt. Of the four hypotheses tested to explain the coincident rise in the VOC and Rt, the observed data were most consistent with the model including increased transmissibility of the VOC relative to pre-existing variants. The authors estimated that the VOC was 56% more transmissible (95% credible interval: 50-74%) than previously observed SARS-CoV-2 variants combined. However, there was no evidence that the VOC led to higher rates of hospitalizations or deaths. Changing contact patterns did not appear to explain the rise in the VOC. Lastly, the authors found that unless schools are closed and vaccinations increased to 2 million per week, the effective reproductive number is unlikely to fall below 1 as a consequence of increased frequency and transmissibility of the VOC.

Study Strengths

This study assessed multiple alternative mechanisms for the rapid rise in COVID-19 cases along with increasing frequency of the VOC in England, including increased transmissibility of the VOC, increased, susceptibility to the VOC among children (which has major implications for school re-openings/closures), and changes in social contact patterns.  Implications of the VOC on vaccination strategies were also explored.

Limitations

Limited information was reported on testing rates or genomic sampling across NHS regions or age-groups over the observation period which may impact interpretation of results. While unlikely, this study does not rule out the possibility that founder and super spreading events with subsequent spread of the VOC to other regions explain the observed data. Of note, there were apparent increases in social contacts among persons <18 years of age immediately prior to the rise in the VOC, which was not addressed in the limitations, despite reported outbreaks among schools over the same time frame. Further, estimates of Rt from contact data appear to fit the data relatively well, except for a brief time period between/during October and November.

Value added

This is the first study to estimate the relative transmissibility of the novel SARS-CoV-2 variant, VOC 202102/01, compared to pre-existing variants and its implications for future transmission dynamics and control.

Our take —

This study used four different analytic approaches to estimate the relative effectiveness of country- and US state-level non-pharmaceutical interventions (NPIs) early in the pandemic. The authors found that social distancing (e.g., cancelling small gatherings), school closures, and travel restrictions (e.g., border closures) were most effective, while environmental measures (e.g., disinfecting surfaces) were least effective. Though each statistical approach was distinct, all relied on a similar set of assumptions, and all were subject to biases. Assessing the effectiveness of NPIs is difficult for many reasons: many were implemented simultaneously, case reporting varies across regions and over time, and the success of a given NPI may heavily depend upon local context. This study is not definitive, but suggestive; it highlights important issues involved with NPI impact evaluation.

Study design

Ecological; Modeling/Simulation

Study population and setting

This study considered 6,068 non-pharmaceutical interventions (NPIs) in 79 territories (countries and US states) implemented in March and April of 2020. The authors ranked the effectiveness of these NPIs in reducing the effective reproduction number of SARS-CoV-2 (Rt) using eight broad themes, and 46 categories of NPIs that were implemented 5 or more times within those themes, established by the Complexity Science Hub COVID-19 Control Strategies List (CCCSL). Four analytic methods were used to rank effectiveness: 1) a case-control study (similar to a difference-in-differences approach), 2) LASSO time-series regression (an approach designed to prevent overfitting of models), 3) random forest regression (a a prediction and classification method) with NPIs ranked by measuring model performance with and without each NPI, and 4) Transformers, a machine learning technique for parallel processing of sequential data that can be applied to time series. All approaches used a range of lag periods (i.e., the timing between implementation of an intervention and its effect on the reproductive number, Rt), and considered possible confounding variables including the duration of the local epidemic, Rt at the time of a given NPI, population size, population density, and the previous number of NPIs implemented (total and within the same category as the given NPI). The results were applied to two external validation data sets.

Summary of Main Findings

There was general agreement among the four methods used to rank NPI effectiveness; of the eight broad NPI themes, social distancing and travel restrictions were the highest ranked, while environmental measures such as disinfecting surfaces were least effective. Six of the 46 categories showed statistically significant reductions in the reproductive number (Rt) with all four methods: canceling small gatherings (change in Rt: -0.22 to -0.35), closing educational institutions (-0.15 to -0.21), border restrictions (-0.06 to -0.23), increasing availability of personal protective equipment (-0.06 to -0.13), individual movement restrictions (-0.08 to -0.13), and national lockdown including U.S. state stay-at-home orders (-0.01 to -0.14). Environmental cleaning, public transport restrictions, and contact tracing measures were among the least effective NPIs across estimating methods. NPI effectiveness was highly heterogeneous across countries. Applying these methods to two external datasets produced broadly similar results. By artificially shifting NPIs to different times in relation to the age of the epidemic, the authors estimated that early adoption was beneficial for lockdown, small gathering cancellations, travel restrictions, and closure of educational institutions.

Study Strengths

This study considered a wide range of disaggregated NPIs, and compared their effectiveness using four distinct methods to examine the sensitivity of NPI ranking to the estimation method. Methods were tested on two external datasets.

Limitations

While the authors employed four different methods for estimating NPI effectiveness, the specification of models in each method was structurally similar, meaning that agreement across methods is less remarkable than it may appear. The estimation of Rt relies on reported cases; case reporting was highly heterogeneous across regions and over time during the study period, and was subject to discrete changes in case definitions and reporting standards. Estimates of changes in Rt may therefore be biased. Also, using this method makes it appear as if measures that increase case ascertainment (e.g., improving testing capacity, contact tracing, etc.) actually increase Rt. The timing of NPI implementation makes identification of impacts difficult, since so many were implemented roughly simultaneously– this issue is exacerbated because the expected lags between implementation and effects are not precisely known. The study is restricted to March and April of 2020, and so territories that were hit earlier by the pandemic are likely overrepresented. Although travel restrictions are ranked highly, they are likely to be effective only during the initial phases of an epidemic when local transmission represents a small share of new cases. In general, the interactions between the age of a local epidemic and NPI effectiveness are not handled well by the approaches here. Finally, although the authors adjusted for several territory-level variables, including the number of previous interventions implemented, residual confounding (e.g., country GDP) and unmeasured interactions (e.g., the success of contact tracing may depend on multiple other factors including public health infrastructure and testing capacity) are likely.

Value added

This is one of the largest attempts to quantify and rank the effectiveness of non-pharmaceutical interventions across multiple regions.  

Our take —

In this study, available as a preprint and thus not yet peer reviewed, authors used a mathematical model to quantify how different testing strategies could be used to reduce the probability of transmitting SARS-CoV-2 to others after leaving quarantine. Any testing during the quarantine period, especially toward the end, contributed to a reduction in the probability of post-quarantine transmission (pPQT). The pPQT was reduced by 98% and 93% for 7-day and 5-day quarantines respectively with PCR testing for SARS-CoV-2 upon entrance and exit into quarantine, as compared to a 3-day quarantine with only entrance testing. Many assumptions in the model, such as an optimistic diagnostic sensitivity and just a 24-hour delay from sampling to returning a test result, may not be realistic or feasible in most settings. However, this work provides a framework to explore how quarantine measures could be adapted to incorporate additional information from test results.

Study design

Modeling/Simulation

Study population and setting

Using a mathematical model, authors explored if different quarantine durations combined with testing at the start and/or end of the quarantine period could have the equivalent reduction in the probability of post-quarantine transmission (pPQT) as the standard 14 day quarantine period with no testing. They explored three scenarios for which quarantine may be needed: ii) for travel regulations; ii) quarantine of contacts identified through contact tracing; and iii) case isolation upon symptom onset. Authors then used a dataset of 4,040 PCR test results from tests administered between April 11 to August 26, 2020 at an offshore oil rig to test their model. They assumed an 8.29 day incubation period and a 24 hour delay between sampling and test results being returned.

Summary of Main Findings

Any testing during the quarantine period contributed to a reduction in the pPQT, with the reduction dependent on the timing of the test and the duration of quarantine. A single test at the end of quarantine of any length consistently resulted in a lower pPQT compared to a single test conducted at the start of quarantine. Optimum time for testing was upon exit from quarantine, day 5, and day 6 for quarantine lasting ≤7 days, 8-13 days, and ≥14 days respectively. In an optimistic scenario with minimal delays to test results, testing at the end (or beginning and end) of quarantine could halve the duration to 7 days. Analysis of the 4,040 PCR test results conducted on an oil rig where workers were tested at the start of a 3-day quarantine, using this framework found that adding a test at the end of a 7 day or 5-day quarantine could reduce the pPQT by 98% and 93% respectively. Authors estimated that 9 offshore transmission events could have resulted in the absence of testing on exit.

Study Strengths

Authors consider several real-world scenarios where quarantine is currently implemented and explore several scenarios for testing and quarantine duration. Their framework is then validated using a large dataset of 4,040 PCR test results conducted amongst employees of offshore oil rigs. Authors also tested if the duration an individual is infected but less likely to infect contacts had an impact on their results.

Limitations

Authors assumed that the incubation period is fixed at 8.29 days which may be unrealistic given the mean incubation period estimated from multiple studies is shorter at 5 – 7 days. The incubation period can also vary between individuals which the authors do not account for in this study. Authors also extrapolated test sensitivity estimates early after contact exposure based on data from hospitalised patients; the diagnostic sensitivity assumptions appear quite optimistic relative to other available data based on more comprehensive data from clinically diverse patient populations. Given their study focuses on the use of testing for release from quarantine we would expect this would also include asymptomatic and/or mild individuals where test sensitivity may differ substantially. Finally, they also made an optimistic assumption that the delay from sampling to receiving test results is 24 hours which may not be realistic or feasible in many settings. The data set used to validate their framework represents a unique closed population which is not representative of the wider population.

Value added

With many countries adopting testing on arrival or as a prerequisite for travel, this study quantifies the value of testing at the end of the quarantine period in addition to, or in place of testing at the beginning of the quarantine period.

Our take —

This preprint, which has not yet been subjected to peer review, used both case counts and deaths in an age-specific model of SARS-CoV-2 transmission in New York City. The authors estimate that the reproduction number of the virus decreased dramatically from the beginning of March to mid-April, 2020 in response to a collection of interventions that reduced mobility (through stay-at-home orders, school closures, business closures, etc.) and mandated mask use. The authors’ attribution of transmission declines to each of these two types of interventions, however, is dependent on many assumptions and subject to much uncertainty.

Study design

Modeling/Simulation, Other

Study population and setting

This study estimated the impacts on SARS-CoV-2 transmission of non-pharmaceutical interventions (NPIs) in New York City (NYC) from March 1 to June 6, 2020. COVID-19 cases included all laboratory-confirmed cases reported to the NYC Department of Health and Mental Hygiene, and deaths combined probable and confirmed deaths associated with COVID-19. Mobility data, used as a proxy for contact rates, were obtained from Safegraph and consisted of anonymized, aggregated counts of visitors (measured by mobile phone location) to locations within each ZIP code. The authors used a neighborhood-specific SEIR network model fit to cases and deaths, stratified by age group, to estimate the effects on transmission of 1) all NPIs, 2) contact-reducing NPIs (such as stay-at-home orders, business closures, etc.), and 3) mask use. Mask use was assumed to explain the reduction in estimated transmission rate that was not accounted for by mobility declines during periods when face coverings were mandated in public places. Model projections beyond the end of the study period were compared to observed cases and deaths.

Summary of Main Findings

Observed, diagnosed COVID-19 cases displayed different age-specific patterns compared to model estimates of underlying infection rates: estimated infection rates were highest for those aged 25-44 years and 45-64 years, and rates for all age groups peaked the week of March 22 or one week later. During the first week of the NYC epidemic (beginning March 1), the estimated time-varying reproduction number (Rt) was 2.99, decreased to 1.37 after the stay-at-home order on March 22, and reached a minimum of 0.56 during the week of April 12. Mobility reductions (a proxy for contact rate reductions arising from stay-at-home mandates, school closures, and other contact-reducing interventions) were estimated to result in a 70.7% (95% CI: 65.0% to 76.4%) decline in Rt by the week of April 12. Assuming that effectiveness of mask use would account for the difference between estimates using a) a linear regression with mobility data alone and b) the SEIR model, the authors estimated that mask use reduced the transmission rate and infectious period by 3.4% (95% CI: -1.9% to 8.6%) over eight weeks, with higher effectiveness during the first month. Estimated mask effectiveness was highest in older age groups and remained stable during the study period (for the first month among those 65-74 years old: 20.8%, 95% CI: -0.1 to 41.6%; 75+ years old: 20.8%, 95% CI: 20.8%, 95% CI: -0.9 to 42.5%). Projections from the week of June 7 through the week of July 26 using parameters based on observed mobility data and estimated mask effectiveness underestimated cumulative cases by 27% and underestimated deaths by 2%.

Study Strengths

The model was fit to both observed cases and deaths, and projections beyond the study period were compared to observed outcomes.

Limitations

This is a preprint, and has not yet been subject to peer review. Aggregated zip-code-level mobility data are an imperfect proxy for actual mobility, which is in turn an imperfect proxy for contact rates. The method used to estimate the effectiveness of mask use relies on strong assumptions and oversimplifications (e.g., all residual reduction in predicted transmission rate after accounting for mobility decline is attributable to mask use; mask use affects both transmission risk per contact and infectious period; dates of mask mandates are a perfect proxy for actual mask use; etc.). Projections did not fit observed data well, which may be an indication that the effect of interventions was overestimated.

Value added

This study provides a useful picture of age-specific patterns of SARS-CoV-2 infection during the spring of 2020 in New York City.

Our take —

Using a simple stochastic transmission model, authors sought to estimate the unobserved SARS-CoV-2 infections in the United States from January 1 through March 12, 2020, as well as projected deaths after March 13 that would have occurred as a result of these infections. The model estimated that fewer than 10% of locally acquired symptomatic infections were detected during this period when surveillance was limited, and that over 100,000 infections actually occurred. The model did not account for geographic differences in cases or deaths, and assumed exponential growth of the epidemic. The estimates were highly uncertain, due to the unpredictable nature of early transmission, which limits the reader’s ability to draw meaningful conclusions.

Study design

Modeling simulation

Study population and setting

Authors sought to estimate the full extent of community-acquired SARS-CoV-2 in the United States from January 1 through March 12, 2020 when testing and surveillance were limited. Using a stochastic simulation model with separate steps for importation of infections and local transmission, authors incorporated data on imported cases and local deaths in the United States, and estimated the probability of detecting daily infections. Authors estimated the total number of infections that occurred through March 12, and then estimated the number of expected deaths after March 13 that would result from those infections.

Summary of Main Findings

There were 1,514 locally transmitted cases and 39 reported deaths in the United States by March 12, 2020. In the baseline analysis, the model estimated that between January 1 and March 12, 45% (95% Posterior Predictive Interval [PPI] 4%, 97%) of imported symptomatic infections were detected; results from the model simulations estimated that 108,689 (95% PPI 1,023, 14,182,310) cumulative infections actually occurred during this time. During the month of February – when authors argue containment measures would have still been possible – the model estimated that fewer than 10% of locally acquired symptomatic infections were detected. The model further estimated that almost 16% of infections occurred on March 12 alone. Among persons estimated infected prior to or on March 12, the model estimated 827 (95% PPI 6, 115,507) additional deaths after March 13 (resulting from the delay from infection to death)

Study Strengths

Overall, results for cumulative infections, local case detection probability, and ratio of deaths after March 12, 2020 were relatively robust to assumptions about serial interval, case fatality risk, timing and probability of early case importations, and delays in reporting. Estimates from the model regarding detection of local symptomatic infections are consistent with previous serological studies.

Limitations

Posterior predictive intervals were quite wide for several estimates, indicating large uncertainty in the model. Although authors intentionally incorporated uncertainty in eight of the parameters used by the model, the width of the intervals makes many of the estimates challenging to interpret. The branching process in the model also assumed exponential growth of infections and did not take into account the effect that social distancing or other factors would have on transmission.

Value added

This modeling study sought to understand the true extent of SARS-CoV-2 community transmission in the United States compared to the number of cases and deaths reported, particularly during early 2020 when testing and surveillance were limited. Projections indicated that surveillance was able to detect fewer than 10% of locally acquired symptomatic cases.

Our take —

This study, available as a preprint and thus not yet peer reviewed, suggests that daily incident PCR positive detections and deaths peaked in Kenya in August 2020, and that cumulative infections are nearing population herd immunity, the theoretical threshold value where there is enough immunity in the population to provide indirect protection to susceptible individuals. The authors developed an innovative model of SARS-CoV-2 transmission that used available seroprevalence, mobility, and PCR testing data to estimate the course of the pandemic. The limitations of the approach, specifically relying on mobility trends and limited seroprevalence estimates, may be overestimating the amount of transmission that has occurred. If this is the case, the stated finding that these locations in Kenya have peaked and reached herd immunity, along with reporting such a notably low infection fatality ratio, should be taken with caution.

Study design

Modeling Simulation

Study population and setting

Authors constructed a data-driven dynamic transmission model to estimate the transmission and seroprevalence of SARS-CoV-2 virus infection in Kenya. The authors developed a simple but effective mathematical model to use the relatively sparse data available to model transmission, using seroprevalence estimates, mobility trends data, and national PCR testing data. In addition to estimating cumulative infections, the authors estimated an effective size parameter, which represents a percentage reduction in the size of the population at risk of infection due to population heterogeneity in connectivity and risk. A smaller effective population size lowers the expected herd immunity threshold, the theoretical threshold value where there is enough immunity in the population to provide indirect protection to susceptible individuals.

Summary of Main Findings

Forty-point-nine percent of Nairobi (95% CI, 24.3-54.7%), 33.8% (95% CI, 23.7-46.5%) of Mombasa are estimated to have been infected with SARS-CoV-2 by August 1, 2020. Authors inferred Nairobi’s effective population size to be 77% of the total population, while Mombasa’s was inferred to be 56% of the total population. These estimates suggest that the epidemic has largely peaked across Kenya, with Nairobi peaking in late July/early August 2020, Mombasa peaking in mid-June, and the rest of the country peaking between August and September. The authors also assume, based on the estimated effective population sizes, that Kenya is nearing herd immunity. The infection fatality ratio in Nairobi was estimated to be 0.014% (95% CI, 0.010-0.023%) and 0.02% (95% CI, 0.014-0.028%) in Mombasa, contrasting with the age-adjusted expected estimate of 0.26%.

Study Strengths

This study developed an innovative yet relatively simple data-driven model for using multiple sources of available data to characterize the pandemic in Kenya. The authors validated the transmission estimates against reported deaths, demonstrating good fit.

Limitations

The model relied on a strong assumption that infectious contact rates were fully correlated with mobility as estimated by Google mobility trends. Thus, their estimates of the time varying reproductive number (Rt) correlated fully with these mobility estimates. This is particularly problematic because transmission has been demonstrated to not fully correlate with increasing mobility trends, as people have improved their ability to limit transmission through social distancing, masks, and other means. As a result, the authors may be overestimating transmission following lockdown in Kenya, resulting in an overestimate of infections and seroprevalence. Additionally, this model is highly reliant on estimates of seroprevalence at two time points from a survey that took place in May and early June. Because of this time gap and limited population representation, it is very possible these two estimates are not generalizable and too long ago to truly inform infection later in the summer and across these populations. Finally, the authors do not account for COVID-19 death underreporting. While the authors state this as a limitation, their estimate of IFR is grounded in this assumption, thus the IFR is very likely underestimated.

Value added

This study provides the first complete assessment of SARS-CoV transmission for Kenya and one of the first for a country in Africa, where substantial questions remain as to why the pandemic has progressed differently than originally expected. This study cleverly combines mobility, seroprevalence, and PCR testing data to estimate transmission and seroprevalence, and provides an important and useful blueprint for doing this in other countries where sparse data have limited the understanding of the pandemic.

Our take —

This study, available as a preprint and thus not yet peer-reviewed, demonstrated that the effectiveness of local lockdowns on COVID-19 transmission are dependent on the duration of intervention and the spillover effects from neighboring municipalities using municipalities in Chile as a case study. Due to spillover events, local control by itself may not be sufficient to control epidemic growth. Greatest reductions in transmission and case numbers could be achieved by extending the lockdown period of the target and neighboring municipalities.

Study design

Modeling/Simulation

Study population and setting

Authors used the daily time series of COVID-19 cases reported by the Ministry of Health of Chile between March 1 and mid-July, 2020 and adjusted this by the delay from symptom onset to report to reconstruct the epidemic curve by date of onset. Census data and household survey data were also incorporated. Transmission in each municipality was characterized by the instantaneous reproduction number allowing for interactions between neighboring municipalities as different municipalities were subject to different localized lockdown policies. Demographic and economic factors that differed by municipality and could affect transmission such as age, sex, sanitation infrastructure, overcrowding, and monthly income were accounted for. The authors used a causal inference framework to explore the impact of counterfactual lockdown policies of different durations.

Summary of Main Findings

Authors found that the effectiveness of local lockdowns are highly affected by the duration of the local lockdown and the level of spillover from neighboring municipalities that may have different levels of transmission and under different control measures. In three municipalities of Greater Santiago a local lockdown lasting 3-weeks longer would have decreased the reproduction number further, preventing 143 per 100,000; 59 per 100,000; and 267 per 100,000 population in Lo Barnechea, Providencia, and Santiago respectively, representing a 33-62% reduction in reported cases in that time period. Authors estimate that these reductions would have been even greater had the neighboring municipalities also extended their local lockdowns.

Study Strengths

Authors used an integrated dataset of daily reported COVID-19 cases adjusted for reporting delays, household survey, and census data to reconstruct the epidemic course in Chile. The natural variation in local lockdowns implemented in each municipality with varying durations were used to explore their effectiveness in reducing transmission. Authors also accounted for the interconnected nature of neighboring municipalities and explored varying counterfactuals and their effect on the epidemic.

Limitations

As data by date of symptom onset were not available, authors had to reconstruct the epidemic curve accounting for the onset to reporting delay to estimate the reproduction number over time. Information on testing efforts are not provided so it is difficult to determine whether the increase in case numbers, particularly early in the epidemic are a true increase in transmission, or an increase in case finding.

Value added

This study provides insights into the effectiveness of local lockdowns and the importance of considering the spatial interconnectedness of neighbouring municipalities and their local interventions.

Our take —

This perspective piece provides evidence that the true prevalence of antibody response against SARS-CoV-2 in the general population may be significantly underestimated, implying more people have been exposed to SARS-CoV-2 than current testing demonstrates due to sub-optimal serologic assay sensitivity in diagnostic testing. Assays require optimization to detect lower antibody titers from patients with mild infections or to account for significant waning of the immune response over time. Modeling assumptions made in this simulation study may not reflect the true test, transmission and immune response in the population.

Study design

Modeling/Simulation

Study population and setting

FDA-approved serologic assays used to monitor seroprevalence of antibody against SARS-CoV-2 often use samples from patients with severe symptoms and recent infections as positive controls in assay validation and optimization. Samples from these patients have high antibody levels compared to subjects with mild or no symptoms and subjects with less recent infections, where antibody levels are expected to decline over time. The use of these samples in assay validation may result in significant spectrum bias, which overestimates assay sensitivity in the general population leading to underestimates of the actual number of people who have been exposed to SARS-CoV-2. This modeling study aimed to quantify the amount of bias introduced in estimating SARS-CoV-2 seroprevalence in the general population due to potentially inappropriate assay sensitivity validation. The authors modeled the potential impacts of varying assay validation sample donor characteristics (proportion with differing symptom severities and/or sample collection time post-infection) on the sensitivity of measuring SARS-CoV-2 seroprevalence in simulations of the general population. Assay specificity was held constant. Simulated study populations were created by differing in the relative proportions of symptom severities (severe, mild, asymptomatic) as well as different times post infection for assaying seroprevalence.

Summary of Main Findings

These modeling analyses imply that current assays lack appropriate sensitivity and thus underestimate the true seroprevalence of antibodies against SARS-CoV-2 in the general population. Using a simulated model set where 95% of all infections were asymptomatic or mild and harbor lower antibody levels than assay validation samples, the authors demonstrated that the true assay sensitivity in the general population of current assays may be as low as 54%. Additionally, most samples used for assay validation are collected within 60 days of infection. Since evidence indicates antibody levels decline over time this also implies that as time since infection increases, the sensitivity of the assays will be reduced, further contributing to an underestimate of the seroprevalence in the general population.

Study Strengths

With respect to sensitivity analysis related to symptom severity, the composition of one simulated data set is 95% asymptomatic carriers and patients with mild symptoms, which aligns closely with the true clinical outcomes of SARS-CoV-2 infection.

Limitations

Several assumptions were made in this analysis, including that assay sensitivity would be greatest for recent severe infections, as well as the transmission model and immune response kinetics, which may not completely represent the true assay, transmission and immune response in the population.

Value added

This modeling simulation provides evidence that serum samples from individuals more representative of the disease spectrum within the general population should be used in SARS-CoV-2 assay validation. The article also highlights the potential benefit of the creation of a reference serum bank for universal SARS-CoV-2 serologic assay validation.

Our take —

This study, available as a preprint and thus not yet peer reviewed, evaluated the impact of 13 different non-pharmaceutical interventions on the time varying reproductive number across 130 countries and territories through June 2020. Impact was assessed using a wide range of models with different time lags (i.e., time frame from implementation of intervention to its impact on transmission) and at varying levels of intensity of the interventions. School closures and internal movement restrictions showed evidence of impact across all models, and thus had strong evidence of effectiveness. However, these results should be interpreted cautiously as multiple interventions were often implemented at the same time, making it difficult to fully disentangle the impact of any single intervention from the others.

Study design

Ecological, Modeling/Simulation

Study population and setting

This study used the implementation dates of a large variety of non-pharmaceutical policy interventions and assed their relationship with COVID-19 time varying reproductive number, Rt, defined as the mean number of secondary cases that one index case will infect at time t) across 130 countries and territories between January 1 and June 22, 2020. Data on non-pharmaceutical COVID-19 policies, categories, implementation dates, and a general index of strength of COVID-19 policy response were obtained from the Oxford COVID-19 Government Response Tracker. Estimates of the Rt across regions and time were from EpiForecasts.

There were four primary analyses. The first main analysis used a statistical regression model to characterize how different types of NPIs were rolled out over time, such as when they occurred and in what order. The second model observed how those NPIs were clustered in time (i.e. what policies were more likely to occur together with what other policies, forming a group or a cluster of policies). A third model attempted to determine the time lags between policy implementation and their effect on Rt by assessing 3 different time lags (1, 5, and 10 days) and estimating goodness-of-fit statistics for each model. Finally, the study used panel regression models to attempt to disentangle the impact of different types of NPIs on Rt accounting for different time lags and the effort at which NPI was implemented (any effort and maximum effort).

Summary of Main Findings

First, the study finds a major increase in NPI intensity across the world in mid-March, followed by a slow reduction in the stringency of interventions. Second, the study finds substantial evidence that NPI policies were more likely to be rolled out in particular orders, typically clustered together in time, depending in part on how policy intensity is defined (any effort or maximum effort). Thirdly, the study found support for lag times between policy implementation and Rt impact between 1-10 days. Finally, the study finds evidence that school closure and internal movement restrictions, and high-intensity public events cancellations and restrictions on gatherings. There was some evidence for impact for workplace closure, income support and debt/contract relief. Evidence for impact was inconclusive for stay-at-home requirements, public information campaigns, public transport closure, international travel controls, testing, and contact tracing.

Study Strengths

This study critically examines and demonstrates how NPI policies are related to each other, and assesses lags in their impact. These are issues that are critical both to this analysis, and many other policy analyses which often do not acknowledge that NPI policies are highly correlated with respect to timing of implementation, and that interventions have lagging effects, due to changing compliance over time. The study uses well-vetted data appropriate for policy impact evaluation. The discussion section contains a frank and well-written interpretation of the results and the interpretable limitations thereof. We find that the findings that NPIs overall had substantial impact on Rt to be relatively robust.

Limitations

While the discussion section involves cautious interpretation of impact of individual NPI because they were implemented at similar times, the highest impact section (the abstract) does not heed that caution, and strongly implies that they identified which specific types of NPIs were most effective despite temporal correlation. The key difficulty – one examined and partially established in the paper itself – is that these interventions are temporally related to each other, and also have time lagged effects in similar timescales as the policy rollouts. One major issue is that there appears to be substantial limitations in the way in which lagged effects were measured. Lags were assumed to be at a fixed amount of time and the same for all policies, but in reality policy lag effects are spread out over a wide period of time, and will be different for different policies in different situations. This, in addition to the close clustering in time between policies, makes it difficult to conclude which NPIs had the strongest evidence for impact.

Value added

The study shows strong evidence that NPI policies are related to each other over time and how that creates difficulty in examining their impact.

Our take —

Using previously published models, authors quantified the degree to which COVID-19 transmission could interrupt malaria prevention services across sub-Saharan Africa (SSA), and the impact these interruptions could have on malaria deaths. The estimated effects varied according to the duration of interruptions, timing of disruptions and malaria seasonality, and how recently routine vector-control measures were implemented. Across SSA, authors estimated malaria deaths could increase by 696,000. However, due to large uncertainties in the models concerning COVID-19 transmission and how countries will respond, estimates should be considered illustrative rather than true projections.

Study design

Modeling/Simulation

Study population and setting

This study quantified interruptions to malaria preventive services due to COVID-19, and the potential impact of these interruptions on malaria morbidity and mortality in Nigeria and across sub-Saharan Africa (SSA). COVID-19 trajectories were constructed using a previously developed age-structured Susceptible-Exposed-Infectious-Susceptible model. Malaria deaths were estimated using a previously published malaria transmission dynamics model. Assuming an R0 (basic reproductive number) of 3, authors assessed the four COVID-19 transmission scenarios: 1. unmitigated (no direct action, but contact rates reduced by 20%); 2. mitigation (isolation and social distancing reduced contact rates by 45% for 6 months); 3. indefinite suppression (strict interventions reduce contact rates by 75% and are maintained indefinitely); and 4. suppression lift (strict interventions reduce contact rates by 75%, but these are only maintained for 2 months, after which time contact rates return to 80%). Authors included varying combinations of interruptions to the following malaria prevention activities: 1. distribution of long-lasting insecticide-treated nets (LLIN) would be delayed for a year or continue as normal, 2. seasonal malaria chemoprevention (SMC) would be interrupted, reduced, or continue as normal, and 3. clinical treatments would be interrupted, reduced, or continue as normal. Using Nigeria as an example, authors also estimated how changes to R0 (reproductive number) of COVID-19 could impact malaria deaths and the effect of broadening target ages for SMC.

Summary of Main Findings

If SMC coverage and case treatment were reduced by 50%, malaria deaths were estimated to increase by 42,000 (95% Uncertainty Interval: 22,000-62,000) in Nigeria and by 200,000 (95% UI: 115,000-285,000) across SSA, even if LLIN campaigns were not interrupted. However, if LLIN campaigns were also interrupted in addition to SMC and treatment reductions, excess malaria mortality was estimated to be as high as 495,000 (95% UI: 296,000-693,000) across SSA. If LLIN campaigns, SMC coverage, and treatment were all interrupted, malaria deaths were estimated to increase across SSA by as much as 696,000 (95% UI: 413,000-978,000). If R0 was 2.5 compared to 3.0, authors estimated the epidemic in Nigeria would increase from 6 to 9 months, thereby increasing malaria deaths by approximately 17%, even if LLIN campaigns continued and some case treatment was maintained. In scenarios where LLIN campaigns were interrupted, increasing the target age for SMC from <5 years to <15 years was estimated to save 22,500 lives in Nigeria.

Study Strengths

Authors conducted sensitivity analyses to account for uncertainty in the models.

Limitations

Overall, results are subject to several sources of uncertainty, the foremost being that it is unknown how COVID-19 will spread across SSA. Countries will likely experience varying levels of transmission and respond with different combinations of mitigation strategies, both of which will affect the degree to which malaria activities are disrupted. Parameters included in the COVID-19 model are based on data from the United Kingdom and China, which may not be appropriate for SSA. Authors used Nigeria as a case example for some estimates, but even these results should not be generalized to other parts of SSA.The model assumed that the impact of COVID-19 on malaria deaths was determined solely by the duration of malaria service interruptions. However, duration is unlikely to be the only contributing factor to the effects of service interruptions on malaria-associated deaths; other factors, such as to what degree services are interrupted (i.e., intensity of interruptions), also likely play a role. It is not clear to what extent routine malaria services have truly been disrupted.

Value added

Dual epidemics of COVID-19 and malaria could quickly overwhelm already vulnerable health systems in sub-Saharan Africa. This study contributes to the growing body of literature estimating the potential indirect effects of COVID-19 in Africa, and illustrates the need to prioritize malaria prevention methods such as routine distribution of long-lasting insecticide-treated nets.