Sigma Metrics for Assessing Accuracy of Molecular Testing

https://doi.org/10.1016/j.clinmicnews.2015.06.001Get rights and content

Abstract

For any diagnostic testing, the bar for accuracy is high, for good reason. Disease status and therapy decisions are often defined by a single positive or negative result. In the case of molecular viral load testing of patients with hepatitis or human immunodeficiency virus infections, expensive antiviral therapies are continued or terminated based on laboratory results. If these results are not accurate, patients may be subjected to additional testing or to treatments, which can add expense, worry, and in some cases medical harms. Viral load cutoff points for therapeutic decisions have been recently lowered; therefore, test result accuracy near the assay's lower limit of quantitation is even more important today than in the past. How can molecular diagnostic laboratories improve the assessment of their viral load assay's accuracy? Answers can be found in the statistical assessment of results derived from testing quality control material. The statistical tools are collectively called Sigma metrics. This review will detail the aspects of Sigma metrics that relate to viral load testing, and also review recent literature for common viral load testing as it relates to Sigma metrics.

Section snippets

Why Do Quality Metrics Matter?

The major assumption physicians make prior to their clinical decisions is that the laboratory's test results are valid – that there is no medically important error that obscures the true result, generating either a false-positive or a false-negative result. Laboratory medicine embraces the paramount importance of quality. With a long history of quality and accuracy for diagnostic testing, it is easy to assume that a test result has the appropriate accuracy, but how can we ensure that the

How Do We Know That When We Achieve the Proper Level of Quality?

The core requirement of any method used for quality assessing is the ability to numerically define what we consider good performance and acceptable quality; therefore, we must also define unacceptable quality and poor performance, i.e., we must define an unacceptable error, i.e., a defect in the method's performance. For many diagnostic tests, the definition of what constitutes acceptable quality is implicit, but not often clearly articulated in an analytical context. For viral loads, such as

A “Traditional” Approach to Quality: Six Sigma

The quality management techniques collectively known as “Six Sigma” have been practiced in health care for several decades, and for even longer in manufacturing, business, and industry (3, 5, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17). The implementation of Sigma metrics has mostly occurred in laboratory subspecialties, such as chemistry, hematology, and immunology.

The core concept of Six Sigma is to identify defects (false negative or false positive reactions) and then reduce or eliminate as many

Defining Quality Requirements for Six Sigma Performance

Most molecular laboratories are not fully accustomed to discussing the “tolerance limits” of our test methods. Laboratorians are far more comfortable discussing a similar concept, known as the allowable total error (TEa). The TEa, the net analytical error is defined as a combination of method imprecision (random error) and method inaccuracy (systematic error, or bias) (16, 17). When a test method exceeds the TEa, the method or process begins to generate defects (result errors), which could be

The Challenge of Adapting Six Sigma to Molecular Methods

For most laboratory methods, broad classification of positive and negative results can be easily defined; however, when one looks more closely at the analytical performance, precision becomes more challenging to describe. Typically, clinically significant cutoffs for HIV and HCV load tests are reported in a logarithmic scale or in international units (18, 19, 20, 21). It therefore requires a change of scale in order to import and adapt the traditional tools of statistical quality control. It

The Real-World Importance of Six-Sigma Quality for HIV Viral Loads

For HIV and HCV testing, there are test utilization scenarios with immediate, profound clinical and financial consequences. Therapies may be terminated or extended and patients may receive more or less care based on viral load test results. The clinicians and patients assume that the quality of the assay allows for the correct answer to be delivered with just one test, but given the new medical decision points expected for some viral load assays that assumption that should be confirmed through

The Real-World Importance of Six-Sigma Quality—HCV

In a study by Wiesmann et al. (28), HCV patients were monitored for a 2-log drop from baseline to ensure therapeutic response. As with HIV load methods, the diagnostic assay precision can directly impact whether or not a patient remains on HCV therapy. As therapies have become effective, the clinical decision points have migrated to the low end of the dynamic range, where accuracy and precision represent challenges for all real-time PCR assays.

Current HCV treatment guidelines state that

Discussion

Molecular assays have been utilized to manage HIV-1 and HCV therapy response for decades. Differences in accuracy and precision at clinical decision points where treatment decisions are being made and patient management is impacted have become a point of laboratory discussion. Historically, laboratories have challenged assay performance only across the full linear range (overall bias); however, there is a shift towards assessing assay performance near or at the clinical decision points. This

Conclusions

Six-Sigma analyses provide a useful tool to assay precision and overall quality for both HIV and HCV load testing. The Sigma metric translates abstract analytical performance characteristics into tangible measures that can impact laboratory operations and patient outcomes. As diagnostic methods and antiviral treatments have evolved, the demands for analytical performance of viral load testing have increased. In previous decades, extreme precision may not have been as necessary, but today's

Disclosure

Danijela Lucic is an employee of Abbott Molecular Inc.; Sten Westgard is a consultant for Abbott Inc.

References (29)

  • F.J. Lee et al.

    Efficacy of initial antiretroviral therapy for HIV-1 infection in adults: a systematic review and meta-analysis of 114 studies with up to 144 weeks' follow-up

    PLoS One

    (2014)
  • M.A. Shaheen et al.

    Evidence-based consensus on the diagnosis, prevention and management of hepatitis C virus disease

    World J. Hepatol.

    (2015)
  • K.E. Blick

    Providing critical laboratory results on time, every time to help reduce emergency department length of stay: how our laboratory achieved a Six Sigma level of performance

    Am. J. Clin. Pathol.

    (2013)
  • D. Burnett

    Collective opinion paper on findings of the 2009 convocation of experts on quality control

    Clin. Chem. Lab. Med.

    (2010)
  • Cited by (5)

    • Statistics for Method Verification of Quantitative Assays in Clinical Microbiology

      2019, Clinical Microbiology Newsletter
      Citation Excerpt :

      Such variability affects the assay's LLOQ. Assays with high precision even at the low end of quantitation can prove to be extremely variable for monitoring anti-retroviral and anti-HCV therapies [21]. It is crucial that assay performance be accurate at the predetermined assay tolerance limit (medical decision interval), defined as the difference between two different results that can be considered significantly different in terms of clinical care.

    • Quantitative Molecular Methods

      2016, Clinical Virology Manual, Fifth Edition
    View full text