Jonathan Schmitz, MD, PhD, D(ABMM)

Among the dramatic effects of the COVID-19 pandemic are the profound changes affecting the field of clinical diagnostics. At the onset of the pandemic, access to—and questions about—testing were among the most vexing challenges facing patients and providers. As a result, hundreds of SARS-CoV-2 assays have been brought to market over the past 3 years, most in the United States under an emergency use authorization (EUA) from the FDA.1 Even now, as the pandemic’s intensity continues to subside, the diagnostic landscape remains profoundly altered. Of note, this holds true for pathogens other than SARS-CoV-2, as the pandemic has underscored subtle—yet critical—concepts that are broadly applicable for diagnosing infections. These concepts are not unique to SARS-CoV-2, and often preceded this virus, although the pandemic escalated them from esoteric conversations among specialists to front-page news.

This article serves as an opportunity to highlight several overarching diagnostic lessons from COVID-19 testing, as well as their broader relevance to other hot topics in the field. These issues range from nuanced details of laboratory analysis to broad concepts in clinical interpretation and laboratory regulation, which showed how profoundly the pandemic has affected all aspects of healthcare delivery!

Lesson 1. Sense and Sensitivity

As soon as RNA sequence information became available for SARS-CoV-2, a variety of molecular assays were developed based on the polymerase chain reaction (PCR) and other nucleic acid amplification techniques (NAATs).2,3 In the United States, the FDA rapidly promulgated initial guidance about how molecular assays could be validated, including how to address requisite performance characteristics through contrived specimens, given the lack of other validation materials at the time.4 In one sense, the progression over mere weeks—from an unknown virus to the development of diagnostic testing—was a testament to modern molecular biology. At the same time, the urgency of the pandemic demanded that assays be deployed in mass without the ability for traditional clinical trials. As many clinicians and clinic laboratory staff can attest, one of the most frequent clinical questions during the first months of 2020 was this: How sensitive are these various new assays for SARS-CoV-2? This not only includes NAATs but also immunodiagnostic antigen tests, which traditionally have demonstrated lower sensitivity for respiratory viruses (eg, influenza and respiratory syncytial virus [RSV]).5 But the reality of emergency testing highlighted an important nuance for all these assay types: “Analytical sensitivity” and “diagnostic sensitivity” are not identical concepts.

Analytical sensitivity reflects a test’s ability to detect a target at defined levels within a particular clinical matrix (eg, a nasal swab in transport media, saliva, bronchoalveolar lavage fluid). Validation typically is assessed by limit-of-detection measurements, which are feasible—even if imperfectly—through contrived specimens. Higher analytical sensitivity implies higher diagnostic sensitivity, but the latter concept is also a function of the biology of the disease. Diagnostic sensitivity represents the ability of a test to identify the presence of an infection itself, relative to some accepted laboratory or clinical gold standard.6

image

While it is possible to define the analytical sensitivity of a test for a novel pathogen (eg, 1 copy/mL in swab-inoculated transport media), its diagnostic sensitivity is also a function of the microbial burden at that anatomic site, as manifested across populations of infected individuals. Clinical knowledge and real-world predicate specimens are required to determine this parameter rigorously. In theory, a molecular test may even suffer from poorer analytic sensitivity without sacrificing much diagnostic sensitivity, for instance if natural pathogen loads trend sufficiently high without compromising the categorical results (detected or not detected).7

As the pandemic unfolded, data became available demonstrating that—atypical cases notwithstanding—the burden of SARS-CoV-2 in upper respiratory specimens is sufficiently high, such that the analytical sensitivity of most NAATs ensure sufficient diagnostic sensitivity.8 Instances still occurred, however, where the FDA-authorized parameters of certain tests were adapted to reflect emerging data about the relative sensitivity of different specimen types (ie, direct nasal/nasopharyngeal swabs vs swabs diluted in transport media).9 And, as with influenza and RSV, antigen assays for COVID-19 inherently possess lower analytical sensitivity, which can translate to occasional missed diagnoses.8

Outside of COVID-19, these same factors remain broadly relevant for the development and interpretation of other microbial diagnostics. For example, the detection of circulating microbial DNA/RNA in the bloodstream can be used to diagnose infections throughout the body. This includes (increasingly) nontargeted metagenomic/agnostic approaches that use next-generation deep sequencing.10 The negative predictive value for any such test is fundamentally linked to this relationship between analytical and diagnostic sensitivity. Particularly important is the expected presence and quantity of microbial nucleic acid within the blood, for both a particular pathogen and the anatomic nidus of infection (ie, what infected tissue is spilling over into the blood). Overall, the analytical sensitivity of a test to detect a circulating nucleic acid must be greater than the expected bloodstream concentration of nucleic acid for each organism/source. Given the extremely diverse number of pathogens that such assays may detect—and potential methodical variations among different assays—discerning these relationships in detail is a significant endeavor, although necessary for such a test to rigorously rule out infection.11

Lesson 2. It’s in the Way That You Use It

The counterpart to diagnostic sensitivity is the concept of diagnostic specificity and the ability of a positive result to accurately demonstrate the presence of disease.6 Here, too, the pandemic has illuminated an important general principle: A test of diagnosis is not the same thing as a test of cure. An assay’s diagnostic specificity inherently is linked to the motivation for that test and the clinical conclusion one hopes to draw from it. A common practice throughout the pandemic has been to follow an initial SARS-CoV-2 detection with a subsequent test of cure several days later, so that a negative result might indicate when a patient is no longer contagious. For many COVID-19 patients, however, it is not uncommon for the results of NAATs to remain positive for an extended period after the appearance, and even resolution, of symptoms. These NAATs may detect low levels of residual viral RNA within a respiratory specimen, even after the pathogen no longer remains viable, producing the same “detected” result regardless.12 The high analytical sensitivity of these tests can, paradoxically, reduce their diagnostic specificity in the context of resolving infection.

Quarantine guidelines for SARS-CoV-2 have now evolved to account for these molecular ghosts,13 but similar scenarios may occur for any number of other infections. Of note, in vitro diagnostic (IVD)-approved indications for many molecular assays, including multiplex syndromic panels, cover only the initial diagnosis of patients for whom an infection is suspected. These panels were not designed as tests of cure, and clinical data often are lacking about their ability to serve in this capacity for their various targets. Indeed, codetection of multiple agents is not an uncommon finding with syndromic panels, especially in populations at risk for frequent infections (eg, upper respiratory infections among children in day care), although differentiating active and resolved processes among multiple positive targets can be formidable.14

By the same token, a test for a symptomatic infection may carry significantly different implications than a screen for asymptomatic carriage. It is true that molecular assays have been developed and validated for asymptomatic screening purposes, most notably in the evaluation of various sexually transmitted infections (STIs).15 However, all assays are not interchangeable in symptomatic and screening roles. Diarrheagenic pathogens, for instance, include several opportunistic bacterial species that also can colonize the gut,14 and which do not need treatment in the absence of symptoms. By testing outside the context of suspected symptomatic infection, the predictive value of a positive result is confounded. For example, prominent debates have surrounded Clostridioides difficile, with concerns that NAATs can possess too high of an analytical sensitivity for detecting clinically insignificant levels of colonization. Algorithmic approaches have been adopted in which positive molecular results for C. difficile, even in symptomatic patients, are followed by an antigen test for clostridial toxin (analytically less sensitive) to evaluate for a causative link between organism and disease in such patients.16

Returning to respiratory infections, while the detection of COVID-19 in an asymptomatic individual prompts quarantine, the presence of other respiratory viruses in asymptomatic individuals is not uncommon, although the proper clinical response can remain poorly defined.17

Lesson 3. Everything Is Relative

From the early days of the pandemic, a proposed strategy to identify active COVID-19 (vs molecular artifacts) has been the measurement of respiratory viral loads (VLs).18 Numerous diagnostic assays for SARS-CoV-2 use real-time PCR (qPCR), providing the inherent potential for quantification through the resultant cycle threshold (Ct) values. These values typically decline in the days after COVID-19 symptom onset, together with the virus’s ability to infect others.19 Cycle threshold values might, therefore, serve as a metric for gauging active infection, particularly for immunocompromised patients or atypical clinical presentations. Similar strategies, in fact, have been suggested for other pathogens, including the C. difficile example cited above.20

Despite these potential capabilities, Ct values are (in themselves) a relative and semiquantitative metric, dependent on the specific PCR reaction and platform. They are 1 step removed from absolute VLs, which require external calibration across an assay’s dynamic range. Several cautionary studies have demonstrated that raw Ct values for SARS-CoV-2 specimens can vary significantly among assays, an important caveat for their clinical interpretation.21 Of note, for virtually all SARS-CoV-2 qPCR assays in clinical use, Ct values have not been validated as part of the clinically reportable qualitative result. A further confounder is the variable units by which viral concentrations are defined across different COVID-19 assays. For various pathogens, metrologic harmonization is facilitated through the international unit (IU) standards by the World Health Organization.22 While IU-based standards have been promulgated for SARS-CoV-2,23 the vast number of NAATs in clinical use were developed with limits of detection alternatively defined in units that include copies per milliliter, genome equivalents per milliliter, and TCID50 (50% tissue culture infectious dose) per milliliter. Such variability can generate uncertainty during attempts to compare performance among assays, whether for SARS-CoV-2 or other pathogens.

For such reasons, ensuring comparability between VL assays has been a long-standing focus for bloodstream viral pathogens that routinely undergo diagnostic quantification. An emphasis on standardization and proficiency testing—including within initial clinical trials—has improved harmonization for many such viruses (eg, HIV, hepatitis C virus [HCV], HBV), although challenges remain and these resources/data are not universally available for all pathogens and specimen types.24,25 This is especially true for more esoteric quantitative assays, whose only availability may be through locally validated assays at individual laboratories.

Lesson 4. Location, Location, Location

As seen from walk-in clinics to pharmacies and grocery stores, diagnostic options for COVID-19 outside of central laboratory environments have become commonplace. These include antigen-based kits for at-home use, as well as both antigen assays and rapid NAATs for point-of-care (POC) analysis by providers.26 While these options represent the new normal, they are a far cry from the more limited diagnostic landscape for SARS-CoV-2 in 2020. This desire for rapidity in diagnosing COVID-19 stems from the ability of such testing to guide patient-management and quarantine decisions in real time. In this same manner, POC antigen assays for several other respiratory pathogens (influenza and RSV) were already commonplace long before the pandemic, and the first NAAT-based POC options for these viruses also emerged before COVID-19.27 However, the past 3 years has further crystallized the desire to bring even more infectious disease testing under POC and self-testing paradigms.

image

Current research and development pipelines, and recently launched assays, include numerous examples of additional pathogens and infection scenarios where noncentralization is a key priority. Additional respiratory pathogens,28 along with STIs29 and urinary tract infections30 represent potential targets. In each case, these conditions are associated with a prevalent epidemiological burden in routine community practice, with a traditional latency period of several days to achieve an in-lab diagnosis. The possibility of broader testing options looks to transform this dynamic, although these new diagnostic possibilities raise practical questions similar to those the field grappled with for COVID-19: What are the actionable benefits of rapid diagnostic capabilities? And can studies demonstrate these advantages in an evidence-based manner?

This question points to a broader trend in rapid diagnostics and clinical microbiology in general: a desire for outcomes research that demonstrates the differential effect of new testing on patient management and financial implications.31,32 It is one thing to present a hypothetical argument about how a new rapid test could lead to improved healthcare outcomes, but studies demonstrating these benefits are more desirable. In going beyond basic analytical performance, such data can guide direct providers, health systems, and payors when optimizing diagnostic policies. Almost paradoxically, a double-edged sword around many new technologies is that, increasingly, the analyses themselves can seem less formidable than gauging their proper utilization.

Lesson 5. Never Just About the Science

The above examples highlight the diverse technical and clinical challenges related to COVID-19 diagnostics. But as the last point indicates, this unprecedented need for this testing emerged against a complex background of broader trends and pressures: logistical, financial, regulatory, and legal. In fact, some of COVID-19’s most challenging diagnostic lessons involved issues other than the science/medicine itself. Among clinical microbiologists, for instance, one of the most dreaded words in 2020-2021 was allocation, as the pandemic revealed potential vulnerabilities in the global supply chain for molecular reagents.33 For many, this period also demonstrated the inherent labor-market inflexibility for specialized staff who can perform molecular testing,34 as well as the critical (and interconnected) need for automation in accommodating periods of extreme clinical demand.35

Within the United States, the first months of the pandemic recapitulated one of the more contentious topics in laboratory medicine over the preceding decade: the ability of individual clinical laboratories to implement their own laboratory-developed tests (LDTs) without FDA oversight. American diagnostic testing is subject to the Clinical Laboratory Improvement Amendments (CLIA), federal legislation that stipulates the performance characteristics that must be defined for diagnostic assays (specifically: accuracy, precision, reportable range, and reference intervals).36 For commercially distributed tests, validation of these parameters, along with others such as analytical sensitivity/specificity, is reviewed by the FDA. But for assays that have not received IVD status from the FDA, individual clinical laboratories must determine/document these parameters prior to local implementation as an LDT. CLIA defines requisite classes of performance characteristics, although it does not dictate specific methodologies for validating them. And while the FDA has long maintained that it holds enforcement discretion over LDTs, tremendous latitude has rested historically with individual laboratories to determine what they consider sufficiently rigorous, with post-implementation inspection by groups like the College of American Pathologists, who possess deemed status for overseeing CLIA certification.6

Returning to early 2020, initial nonbinding guidance from the FDA provided instructions for validating new diagnostics for SARS-CoV-2.4 However, these documents also suggested that clinical laboratories should apply for their own EUAs for any protocols that had not received approval broadly, rather than use the general paradigm of LDT development. Given the lack of commercial testing platforms at this stage of the pandemic, this guidance led to a various single-site EUA assays at commercial and academic laboratories throughout the country.1 Following several months of discourse,37 however, the Department of Health and Human Services (HHS) instead clarified that the ability to regulate lab-specific testing fell outside the legally defined purview of the FDA, reopening LDT options for SARS-CoV-2 to individual accredited laboratories.38 That HHS policy was then further withdrawn over a year later (under the next presidential administration),39 although at a point when the high-throughput commercial options were far more plentiful.

In general, a theoretical advantage of LDTs is their ability to provide testing for clinical conditions where market forces alone do not drive commercial manufactures to undertake the process of test development and FDA submission. Given the extraordinarily diverse nature of microbial pathogens—and the inherent flexibility of NAATs—molecular infectious disease testing represents one of the more common examples of LDTs in current practice. At the same time, LDTs also can create a unique ability to affect patient care without universal thresholds of regulatory or peer review. Moreover, while the FDA considers the clinical validity of a test as part of its pre-review, LDTs only need to be validated for analytical validity under CLIA, with interpretation left to the ordering provider.40 In these ways, LDTs can provide a mechanism for the commercialization of diagnostic tests without universal standards of validation and interpretation, with reference laboratories developing their own in-house LDTs.

Looking ahead, the only thing that seems certain about the future of “home-brew” testing is that it promises to be dynamic. For instance, in recent years, the proposed Verifying Accurate, Leading-edge IVCT Development (VALID) Act would place future LDTs under new risk-tiered categories of in vitro diagnostics that fall under explicit FDA oversight.41 However, the VALID Act is yet to pass both houses of Congress and be signed into law, and its future remains uncertain (with both supporters and critics among prominent professional societies). In the United States, further complexity exists at the state level, for instance within New York, where LDTs already require explicit approval by the State Department of Health.42

Finally, one of the most dramatic potential changes to the American LDT landscape came just before this article went to press, when the FDA announced plans to “phase out” its enforcement discretion of LDTs going forward, with such tests eventually coming under the same enforcement paradigm as other IVDs.43 As of the time of this writing, this proposed rule is within a 60-day public comment period for the medical/laboratory community, after which the FDA plans to announce more definitive next steps. This includes the dynamics of whether/how any existing LDTs might be grandfathered, along with whether LDTs performed by clinical laboratories at academic medical centers (as opposed to commercial entities) warrant their own unique considerations, for example, as a potential middle ground between practices to date and more universal regulation. As brought to the forefront by COVID-19, but hardly limited to the pandemic, this still-raging debate highlights the complex reality of modern diagnostics at the intersection of molecular technology, healthcare economics, and multilayered regulation and enforcement.

Conclusion

The pandemic frequently has been described as a once-in-a-lifetime phenomenon, and it is the first time that such an event has occurred in the age of modern molecular biology and medicine. Obviously, there is now ubiquitous relief that the acute global burdens of SARS-CoV-2 appear to be in the past. But its impact and lessons for clinical practice remain, with the field of diagnostics as a foremost example. As this field continues to advance into unprecedented new places, these lessons will, I hope, serve a grounding effect in exciting but uncharted territory.

References

  1. CDC. Accessed November 26, 2023. https://bit.ly/ 3RksXb1-IDSE
  2. Habibzadeh P, Mofatteh M, Silawi M, et al. Crit Rev Clin Lab Sci. 2021;58(6):385-398.
  3. Christensen BB, Azar MM, Turbett SE. Infect Dis Clin North Am. 2022;36(2):327-347.
  4. FDA. Federal Register. March 6, 2020. 85 FR 13169:13169-13170.
  5. Green DA, St George K. J Clin Microbiol. 2018;56(10):e00711-18.
  6. Burd EM. Clin Microbiol Rev. 2010;23:550-576.
  7. Adams NM, Leelawong M, Benton A, et al. J Med Virol. 2021;93(1):559-563.
  8. CDC. Accessed November 26, 2023. https://www.cdc.gov/ coronavirus/ 2019-ncov/ hcp/ testing-overview.html
  9. Mitchell SL, George KS. J Clin Virol. 2020;128:104429.
  10. Blauwkamp TA, Thair S, Rosen MJ, et al. Nat Microbiol. 2019;4(4):663-674.
  11. Simner PJ, Miller S, Carroll KC. Clin Infect Dis. 2018;66(5):778-788.
  12. Binnicker MJ. J Clin Microbiol. 2020;58(11):e01695-20.
  13. CDC. Accessed November 26, 2023. https://www.cdc.gov/ coronavirus/ 2019-ncov/ hcp/ duration-isolation.html
  14. Dien Bard J, McElvania E. Clin Lab Med. 2020;40(4):393-420.
  15. Tuddenham S, Hamill MM, Ghanem KG. JAMA. 2022;327(2):161-172.
  16. Culbreath K, Ager E, Nemeyer RJ, et al. J Clin Microbiol. 2012;50(9):3073-3076.
  17. Byington CL, Ampofo K, Stockmann C, et al. Clin Infect Dis. 2015;61(8):1217-1224.
  18. Lee MJ. Expert Rev Mol Diagn. 2021;21:1017-1023.
  19. Binnicker MJ. J Clin Microbiol. 2021;59(11):e0046921.
  20. Senchyna F, Gaur RL, Gombar S, et al. J Clin Microbiol. 2017;55(9):2651-2660.
  21. Evans D, Cowen S, Kammel M, et al. Clin Chem. 2021;68(1):153-162.
  22. Baylis SA, Wallace P, McCulloch E, et al. J Clin Microbiol. 2019;57(1):e01056-18.
  23. World Health Organization. Accessed November 26, 2023. https://www.who.int/ publications/ m/ item/ annex-6-trs-no-1004
  24. Hayden RT, Sun Y, Tang L, et al. J Clin Microbiol. 2017;55(2):423-430.
  25. Valsamakis A. E Clin Infect Dis. 2016;63(5):590-591.
  26. Loeffelholz MJ, Tang YW. Detection of SARS-CoV-2 at the point of care. Bioanalysis. 2021;13(15):1213-1223.
  27. Azar MM, Landry ML.J Clin Microbiol. 2018;56(7):e00367-18.
  28. Beal SG, Posa M, Gaffar M, et al. Pediatr Infect Dis J. 2020;39(3):188-191.
  29. Adamson PC, Loeffelholz MJ, Klausner JD. Arch Pathol Lab Med. 2020;144(11):1344-1351.
  30. Weir NM, Pattison SH, Kearney P, et al. PLoS One. 2018;13(6):e0198595.
  31. Doern GV. J Clin Microbiol. 2014;52(5):1314-1316.
  32. Axelrad JE, Freedberg DE, Whittier S, et al. J Clin Microbiol. 2019;57(3):e01775-18.
  33. Hilborne LH, Sossaman G, Caldwell B, et al. Am J Clin Pathol. 2022;158(2):158-159.
  34. Weiss A, Laut CL, Antoine-Reid T, et al. Arch Pathol Lab Med. Published online December 20, 2022. doi:10.5858/arpa.2022-0155-SA
  35. Das S, Frank KM. Clin Lab Med. 2022;42(2):261-282.
  36. Centers for Medicare and Medicaid Services. Federal Register. 1992;57:7002-7186.
  37. Evans BJ, Clayton EW. Yale Law Review Forum. 2020;130:28-100.
  38. Lim D, Brennan Z. Accessed November 26, 2023. https://www.politico.com/ news/ 2020/ 08/ 19/ trump-fda-review-coronavirus-tests-398924
  39. Howard J. Accessed November 26, 2023. https://www.cnn.com/ 2021/ 11/ 15/ health/ covid-lab-tests-hhs-withdraws-policy-bn/ index.html
  40. Center for Medicare & Medicaid Services. Accessed November 26, 2023. https://www.cms.gov/ regulations-and-guidance/ legislation/ clia/ downloads/ ldt-and-clia_faqs.pdf
  41. Hata J, Madej R, Babady NE. J Clin Virol. 2021;141:104875.
  42. Budelier MM, Hubbard JA. J Mass Spectrom Adv Clin Lab. 2023;28:67-69.
  43. FDA. Federal Register 2023;88(190):68006-68031.

About the author:

img-button
Jonathan Schmitz, MD, PhD, D(ABMM), is a clinical microbiologist and an investigator of microbial pathogenesis at Vanderbilt University Medical Center and School of Medicine, in Nashville, Tennessee. For several years, he served as the medical director of its diagnostic Molecular Infectious Diseases Laboratory, where he oversaw development and implementation of SARS-CoV-2 testing for the Vanderbilt Health System. He currently serves as his department’s director of translational microbial diagnostics and co-director of the Center for Personalized Microbiology, and his research group applies techniques and resources of the clinical laboratory environment to elucidate more basic questions of host–pathogen biology.
Dr. Schmitz reported no relevant financial conflicts of interest.

Copyright © 2023 McMahon Publishing, 545 West 45th Street, New York, NY 10036. Printed in the USA. All rights reserved, including the right of reproduction, in whole or in part, in any form.

Download to read this article in PDF document:
Jonathan Schmitz, MD, PhD, D(ABMM)

This article is in PDF format and it requires Abobe Reader. If you do not have Adobe Reader installed on your computer then please download and install from the link below.
Download Adobe Reader