Skip to main content

Test performance evaluation of SARS-CoV-2 serological assays

Our take —

This study provides the first large comparison of lateral flow assays, and demonstrates that while some may be useful at detecting past exposure to SARS-CoV-2 in stored serum/plasma specimens, many had high rates of false-positives which may preclude them from being useful in some settings with low prevalence. Additionally, a larger and better characterized set of samples, including diversity in time since symptom onset and severity of disease/symptoms, is needed to understand how these tests might perform in other populations.

Study design

Cross-sectional

Study population and setting

This cross-sectional study comparatively evaluated the test performance of 10 immunochromatographic lateral flow assay (LFA) kits, the Epitope IgM and IgG enzyme-linked immunosorbent assay (ELISAs) that uses the NP antigen, and one in-house ELISA that uses the RBD antigen to detect anti-SARS-CoV-2 antibodies in stored/remnant serum and plasma samples. Three different sets of specimens were tested by each assay: (1) 130 plasma samples from 80 individuals with RT-PCR-confirmed SARS-CoV-2 infection; (2) 108 pre-SARS-CoV-2 plasma samples from U.S. blood donors; and (3) 52 samples collected from individuals during the SARS-CoV-2 outbreak who were positive for non-SARS-CoV-2 viral infections and/or tested negative for SARS-CoV-2 by RT-PCR.

Summary of Main Findings

Among those with PCR-confirmed SARS-CoV-2 infection, the percentage of specimens that were positive by each LFA generally increased with time from symptom onset and by disease severity. As assessed among samples from pre-SARS-CoV-2 blood donors (i.e., samples that we know should be test negative), the specificity (the proportion of truly negative samples testing negative) of each LFA ranged from 84.3% to 100.0%. The percentage of positive results by the LFAs among persons who had a non-SARS-CoV-2 respiratory viral infection and/or were PCR-confirmed negative for SARS-CoV-2 during the COVID-19 outbreak ranged from 0% to 26.9%. The LFAs supplied by Wondfo and Sure had specificities >99% in the pre-COVID-19 sample set and a 0% false-positivity rate in the ancillary sample set collected from persons who had a non-SARS-CoV-2 respiratory viral infection and/or were PCR-confirmed negative for SARS-CoV-2. There was strong agreement in LFA scores between two readers, particularly for reading IgG bands.

Study Strengths

The study compared the performance of tests on relatively well-characterized specimens, and included an evaluation of inter-operator and inter-lab reliability. Although not described here, the manuscript includes an external validation study.

Limitations

The study did not include specimens from individuals who were infected with SARS-CoV-2 but who were asymptomatic (i.e., all included cases were seeking clinical care). Additionally, specimens from individuals with an extended time since symptom onset were not included in the evaluation. There were also insufficient quantities of some specimens to be tested by all assays, limiting direct comparability. Specificity of the assays was assessed in a relatively small sample size of healthy blood donors, and should be interpreted with caution. Furthermore, results from the third set of specimen collected during the COVID-19 outbreak are difficult to interpret, as 20/52 samples lacked a SARS-CoV-2 RT-PCR test, and of the samples that were confirmed to be PCR-negative, some may have been misdiagnoses of COVID-19 given the poor sensitivity of some SARS-CoV-2 RT-PCR tests. Finally, the performance characteristics of the LFAs described in this study may not be generalizable to their use with capillary blood.

Value added

This study evaluates the test characteristics of a large number of LFAs, and demonstrates considerable variability in test performance. This underscores the importance of additional validation studies, and continued development and improvement of diagnostic serological tests.