Current best evidence for clinical care (more info)
OBJECTIVE: To assess the accuracy of the AbC-19 Rapid Test lateral flow immunoassay for the detection of previous severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection.
DESIGN: Test accuracy study.
SETTING: Laboratory based evaluation.
PARTICIPANTS: 2847 key workers (healthcare staff, fire and rescue officers, and police officers) in England in June 2020 (268 with a previous polymerase chain reaction (PCR) positive result (median 63 days previously), 2579 with unknown previous infection status); and 1995 pre-pandemic blood donors.
MAIN OUTCOME MEASURES: AbC-19 sensitivity and specificity, estimated using known negative (pre-pandemic) and known positive (PCR confirmed) samples as reference standards and secondly using the Roche Elecsys anti-nucleoprotein assay, a highly sensitive laboratory immunoassay, as a reference standard in samples from key workers.
RESULTS: Test result bands were often weak, with positive/negative discordance by three trained laboratory staff for 3.9% of devices. Using consensus readings, for known positive and negative samples sensitivity was 92.5% (95% confidence interval 88.8% to 95.1%) and specificity was 97.9% (97.2% to 98.4%). Using an immunoassay reference standard, sensitivity was 94.2% (90.7% to 96.5%) among PCR confirmed cases but 84.7% (80.6% to 88.1%) among other people with antibodies. This is consistent with AbC-19 being more sensitive when antibody concentrations are higher, as people with PCR confirmation tended to have more severe disease whereas only 62% (218/354) of seropositive participants had had symptoms. If 1 million key workers were tested with AbC-19 and 10% had actually been previously infected, 84 700 true positive and 18 900 false positive results would be projected. The probability that a positive result was correct would be 81.7% (76.8% to 85.8%).
CONCLUSIONS: AbC-19 sensitivity was lower among unselected populations than among PCR confirmed cases of SARS-CoV-2, highlighting the scope for overestimation of assay performance in studies involving only PCR confirmed cases, owing to "spectrum bias." Assuming that 10% of the tested population have had SARS-CoV-2 infection, around one in five key workers testing positive with AbC-19 would be false positives.
STUDY REGISTRATION: ISRCTN 56609224.
| Discipline / Specialty Area | Score |
|---|---|
| Public Health | |
| Respirology/Pulmonology | |
| Infectious Disease | |
| Family Medicine (FM)/General Practice (GP) | |
| General Internal Medicine-Primary Care(US) | |
| Hospital Doctor/Hospitalists | |
| Internal Medicine | |
| Emergency Medicine | |
| Intensivist/Critical Care | |
This was a good study on the evaluation of a rapid antibody test for COVID-19. In this study, the sensitivity and specificity were noted to be lower than in previous (unpublished) results. They also identified a potential overestimation of sensitivity for participants with unknown previous COVID-19 infections, and in a low prevalence population (10%) of a potentially high (1 in 5) false-positive test in ideal conditions.
In the early months of the 2020 COVID-19 pandemic, academic emergency physicians sounded an alarm to guard against the unacceptably insufficient diagnostic science emerging (https://onlinelibrary.wiley.com/doi/full/10.1111/acem.14048), while simultaneously suggesting approaches to mitigate the myriad diagnostic biases associated with poor research methods. This study provides an impressive rebuttal against non-peer-reviewed early reports for one SARS-CoV-2 antigen test. The quantitative impact on observed sensitivity and specificity associated with spectrum effect and imperfect gold standard bias are worth readers' attention.
This is a study of the accuracy of the AbC-19 Rapid Test lateral flow immunoassay compared with the criterion standard of the Roche Elecsys antinucleoprotein assay for diagnosing Covid-19 in English key workers, of whom 9.4% had a positive criterion standard test. Although the authors focus on sensitivity, specificity, and predictive values, likelihood ratios are more useful. The likelihood ratio for positive and negative tests were 44 and 0.08, respectively. Subgroup analysis showed lower sensitivity for unselected populations compared with PCR-confirmed cases of Covid-19, and for asymptomatic patients. For populations with a 10% actual prevalence of disease, around 20% of those with a positive AbC-19 test would be false-positives. One limitation of this study is that the specificity used to calculate the likelihood ratios was measured in a preCovid-19 population. This guarantees that that population did not have Covid-19 but may have resulted in an overestimation of specificity.
I'm not sure how many family doctors will be involved in testing "key workers" in the wake of this pandemic?
Well done diagnostic accuracy study. Interesting but not sure how widely it's applicable.
In this well-designed study, the authors determine the sensitivity and specificity of a COVID antibody test using cohorts of people with PCR positivity and sera from before COVID hit. The average time from PCR to serum test was 60 days, but the range is likely very important: 3.6% of the time, there was no agreement between two examiners on the positivity; the authors point out that their less-than-perfect sensitivity, when used in a low-risk population, would generate a significant number of false-positives. Honestly, beyond its epidemiologic value, I think the clinical value to antibody tests in medical practice is very low. Patients clearly "just want to know," but a key factor in any test is how it is acted on. Patients with positive antibodies are NOT definitively immune to reinfection, and even those who may be now may not stay that way. Tests should be limited clinically to those with unexplained vascular, neurologic, and rheumatologic syndromes, and not just for "curiosity."
Working with COVID patients, we often struggle with the interpretation of the results, especially the antibody testing used previously that had a high 1 in 5 false-positive rate in the general population. We assume there are asymptomatic patients based on this high false-positive rate. Antibody formation means previous infection assumption cannot be made based on this assay, or it should be selectively used in previously known infected population to see whether they developed antibodies.