The validation and verification of laboratory methods and procedures before their use in clinical testing is essential for providing a safe and useful service to clinicians and patients. This paper outlines the principles of validation and verification in the context of clinical human molecular genetic testing.
We describe implementation processes, types of tests and their key validation components, and suggest some relevant statistical approaches that can be used by individual laboratories to ensure that tests are conducted to defined standards. The process of implementing a molecular genetic test for diagnostic use is complex and involves many levels of assessment and validation. The key components of the process, as detailed by the ACCE framework, are analytical validation, clinical validation, clinical utility and consideration of the ethical, legal and social implications of the test.
The development stage involves assessment of both the diagnostic and technical use of the process to ensure that the measurements obtained are relevant to the diagnostic question s and that the analyte s can be unambiguously identified ie, that there are no confounding factors. The final stage of the laboratory process is to determine whether the performance of the test, in terms of accuracy, meets the required diagnostic standards.
Differences between Verification, Calibration and Validation
Whether this is achieved by performing analytical validation or verification depends on the existence of a suitable performance specification that details the expected accuracy of the test under given conditions. The results of the analytical validation or verification determine whether, and how, the test will be implemented and set the requirements for performance monitoring ongoing validation of the test. A simplified process diagram illustrating these concepts is given in Figure 1.
The process of implementing a molecular genetic test for diagnostic use. The shaded arrows represent the two general routes to implementation, depending on the availability of a suitable performance specification: validation lighter and verification darker. Broken arrows represent the situation in which validation or verification fails to meet the specified requirements.
The validation or verification of methods, as defined in Table 1is a formal requirement for the accreditation of laboratories according to the two major international standards applicable to genetic testing laboratories, ISO 2 and ISO Definitions and summarized requirements of the major international standards for accreditation of genetic testing laboratories.
To provide more detailed and specific guidance, Eurogentest 4 set up a working group comprising clinical and laboratory scientists and experts on quality assurance and statistics from both Europe and the United States.
The aims were to develop a framework for validation that could be widely implemented in laboratories to improve the overall quality of genetic testing services while respecting the need for flexibility imposed, for example, by regional requirements and regulations, as well as practical constraints such as test volume and resources.
Test verification and validation for molecular diagnostic assays
In a recently generated parallel initiative, Jennings et al 5 have provided a thorough discussion of FDA regulation, together with a good review of validation procedures. However, specific interpretation of the standards and practical guidance for molecular genetic tests are still lacking. In this paper we propose a generic scheme for the validation and verification of molecular genetic tests for diagnostic use.
This paper is specifically focused on processes involved in analytical validation and verification of tests in human molecular genetics so as to provide working detail of the first component of the ACCE framework.
There is much debate about the exact boundary between development and validation, and good cases can be made for different divisions. For the purpose of simplicity, we have defined a definitive boundary placing all concepts that relate to test utility in development ie, out of scope and parameters relating to test accuracy in validation ie, within scope.
These limitations of the scope should not be taken as assigning different levels of importance to the various processes; making clinically useful and appropriate measurements is clearly critical to setting up a valid diagnostic test. For this reason, we have included a brief section outlining the processes involved and important factors that should be considered at the development stage.
Although we are concerned with appropriate use of statistics and sample sizes, this paper is not intended to be a treatise on the subject, but a practical guide for diagnostic molecular geneticists to aid them in designing, performing and reporting suitable validation or verification for the tests they wish to implement.
References have been provided in which more complex statistical concepts are involved, but it is recommended that the advice of a statistician be sought in case of doubt.Data Scrutiny is important before reporting. You have perhaps come across these terms in laboratory documents and wondered that they convey the same meaning so where is the need for different terminology.
A little insight will help you understand the fine differences between them. Giving their concise definitions may suffice to gain a basic understanding but to help you get a clearer understanding I have attempted to offer clarity on the terms in context of routine activities in analytical laboratories as I believe that giving real life examples rather than recalling definitions serves the purpose better and makes you remember what is being conveyed longer.
In simple terms verification means confirming the authenticity of activities or data before communication to concerned parties. It is important that the results communicated by you are free from errors. Essentially verification comprises of multiple checks on supplies, samples and data before forwarding the results of your testing activities.
Majority of the checks can be carried out by you but it is advised that independent assistance be taken for rechecking the data before submission. It is strongly recommended that independent rechecking be carried out by a set of trained individuals before the analysis results are consolidated for report generation.
All the steps covered above constitute verification before the results are entered for generation of the certificate of analysis. Calibration of laboratory instruments and glassware at stipulated intervals is mandatory for preserving the integrity of reported data.
It helps in making suitable corrections on observed values. Such deviations result from normal wear and tear of instrument components and limitations due to useful life of instrument components.
Calibration is carried out using a standard reference material. The observed values should lie within acceptable range. Periodic calibration is an absolute must for both highly sophisticated instruments as well as for routine instruments such as pH meters, moisture titrators, refractometers, analytical balances, etc. Volumetric glassware needs to be calibrated at least once before being put in routine use.
It is a documented process which establishes and assures that the method or instrument will provide consistent results within the prescribed acceptance criteria. Validation is different from calibration as it does not require comparison with a standard reference material. It is not require to be performed periodically as prescribed in standard operating procedures on calibration but it is sufficient to validate a method at the stage of adoption.
A new instrument, on the other hand is validated at time of installation and commissioning or when important changes are made, for example, addition of a major accessory, relocation to a new premises or when analysis is required under a changed set of environmental conditions. It is hoped that the concepts provided through routine daily laboratory activities will help you appreciate the differences between the terms Verification, Calibration and Validation which are the three cornerstones that define the parameters for sustained growth of laboratories.
Dr Deepak Bhanot is a seasoned professional having nearly 30 years expertise beginning from sales and product support of analytical instruments. His mission is to develop training programs on analytical techniques and share his experiences with broad spectrum of users ranging from professionals engaged in analytical development and research as well as young enthusiasts fresh from academics who wish to embark upon a career in analytical industry.Let me first take a minute to define these terms.
Validation tests how well you addressed the business needs that caused you to write those requirements. It is also sometimes called acceptance or business testing. In a more traditional Waterfall process, with Specifications and Requirements defined at the start, Validation is often performed at the end of the testing cycle.
In an Agile development processideally both verification and validation activities occur as close to simultaneously as possible.Validation/Verification of Multi-target Tests in Microbiology [Hot Topic]
Part of the role of the Product Owner is to define what features should look like ahead of a sprint, elicit any customer feedback needed during development, and speak for the user if necessary. Validation therefore in part falls on the P. Applying Agile Verification and Validation to some industries can be a little intimidating. Verification activities are fairly straightforward and often done via automation. Validation can be trickier. Structure and documentation is inherent in the way the FDA views testing, but structure does not mean that it must be pre-scripted.
An example might be: " Explore drug inventory control With a variety of products To Discover if entering the information is intuitive". Or better yet let them have some fun coming up with a bunch of these themselves. This can work great as a method to uncover defects in unwritten requirements and as a way to test User Experience.
Sample Deployment Checklist. Checklist of Load Testing Metrics. Thanks for Subscribing Keep an eye on your inbox for more great content. Continue Reading. Add a little SmartBear to your life Stay on top of your Software game with the latest developer tips, best practices and news, delivered straight to your inbox.In this first lesson, Dr. Paulo Pereira introduces some of the basic concepts of method validation that apply when a qualitative method is being evaluated.
The validation of qualitative tests differs from the quantitative tests principally since there are no numerical results but binary results, e. Immediately these tests are recognized in medical laboratories according to this designation.
However, they could be related to nominal quantities or ordinal quantities . The validation concepts in this essay only deal with the final binary result that can be applied to any qualitative test. The cases in this lesson use virology results. However, the concepts can be applied to any other qualitative test. Verification and validation definitions are sometimes confusing in practice. In contrast to the verification explanation, validation is directly related to the interested parties requirements, such as the accuracy of clinical decision required by the patients.
Erroneous binary results, i. Therefore, the qualitative test validation goal is to confirm, based on data, that the requirements for its use have been fulfilled. These specifications should be intended to assure a nonsignificant risk of false results.
Reverend Thomas Bayes developed a probabilistic model for defining the likelihood that an element would be a member of a specific class. On a qualitative test view, the element is the binary result, and the class is the disease or nondisease group of subjects. So, the model determines the probability of a patient or a healthy individual to be truly classified as infected or noninfected.
The comparison of methods can be determined primarily when the comparator is the diagnostic accuracy criteria, or it can be determined secondarily when the comparator is other than the diagnostic accuracy criteria. The area under the receiver operating characteristic ROC curve is another diagnostic accuracy measurement, complementary to the sensitivity and specificity .
The patient samples for this validation should be taken from the target population. A representative sampling of infected individuals can be difficult to obtain for a virology qualitative test. The sources of uncertainty in chemistry tests are related principally to the pre  and the analytical phase, to the intra- and inter-individual variation, and to the effects of diseases , drugs  and herbs and natural products . In a virology tests, however, several additional sources variation can affect the accuracy of the results, such as the types and sub-types of the agent, mutations , and the seronegative window period .
Note that the infected individual sampling must only have samples from diagnosed individuals. If the only available samples are from patient samples with a known result from another screening test, this uses the secondary model.
Samples from noninfected individuals are much easier to obtain. Regular blood donors are an example of a suggested population. In this sampling, the target population is the set of generic healthy individuals.
The number of samples is a limitation to the statistical power of the study. As many samples should be used as is practical.So often we see and hear in the news about new laboratory tests that have been developed to detect or manage conditions or diseases that affect our life or that of someone we know.
As with other products and services, new laboratory tests are meant to satisfy a need: to help us and our healthcare providers screen for, diagnose, or monitor conditions faster, easier, and with more confidence. But how does a particular test that shows promise in the research stages actually get to the point where it is available for use at our doctor's office, clinic, or hospital?
What does it mean for you, the healthcare consumer, when a new test is announced or when headlines tout the latest research? How are your healthcare needs met with the advancement of new tests and how is your health protected from new tests that might misinform or mislead your healthcare provider? Becoming familiar with how laboratory tests are navigated through the development, validation, and approval stages and placed into practice may help you understand the answers to these questions and put the latest headline news into appropriate context.
It may take years for a new test to pass through the many phases — research, testing, clinical evaluation, development of manufacturing processes, and review by regulatory authorities — before the test is available for use. It is an intensive process with no assurance that the test, once developed and validated, will actually be adopted by healthcare providers.
That is why the first step is usually determining whether the proposed test will be useful for patients and healthcare practitioners.
Researchers continually look for new ways to improve early detection and diagnosis of diseases, more accurately monitor conditions, and better predict outcomes prognosis. The goals of improving and advancing patient care often provide the incentive for the development and use of new or improved laboratory tests.
One of the most common ways a new test gets developed is through the recognition of a need for an accurate test to diagnose or monitor a particular disease or condition. Measurement of troponin levels in the blood is routinely used by the medical community as a test for evaluating patients with chest pain to help determine if they have had a heart attack. Often, researchers look to improve the way in which a condition is detected or a substance of interest is measured.
Test verification and validation for molecular diagnostic assays.
This can sometimes be accomplished by developing and employing a new way methodology of testing. This can directly impact how quickly a diagnosis can be obtained, how long a patient stays in the hospital, or alter any medications taken. In some cases, an existing test may evolve to have a new clinical use. A test may have been developed for one purpose, but over time, a new use for the test becomes apparent.
Researchers, however, found a new use: measuring it with a highly sensitive testing method that can detect slight increases even when they are within the "normal range" to help determine a cardiovascular event.
There are several questions that can be considered when evaluating the merits of developing a new test:. Though the reasons for developing new tests or methods may vary, it is important to note that the development of all new tests is highly regulated. Each new test must meet certain criteria before it is allowed to be used on patient samples. To understand this process of validation, it is helpful, first, to understand that there are essentially two main kinds of tests:.
These two types of tests take somewhat different paths for validation and approval, but in each case, their development and use on patient samples is governed by sets of rules that ensure their accuracy and reliability.
The Clinical Laboratory Improvement Amendment CLIA 88 is part of federal law that regulates laboratory testing and the quality assurance programs that oversee their validation and use. Professional accrediting organizations such as the Joint Commission and the College of American Pathologists CAP also provide continuing oversight of laboratories and laboratory testing.
There is no assurance that once a test is developed, approved, and available that it will be immediately used by a healthcare practitioner.An EUA allows FDA to authorize use of an unapproved MCM, in anticipation of a potential emergency or during an actual emergency involving a chemical, biological, radiological, or nuclear agent, or emerging infectious disease, if criteria in section of the Federal Food, Drug, and Cosmetic Act are met.
The template for these EUA submissions is available. If you need additional information completing the template or wish to consider use of an alternative specimen type, please contact the Division of Microbiology Devices at or email CDRH-EUA-Templates fda.
This page is intended for laboratories that are implementing in vitro diagnostic assays tests under an EUA and will help laboratories using tests available under EUA learn more about the benefits and limitations of EUA tests, and answer frequently asked questions, including how to report problems with an EUA test to FDA during an emergency.
Patients, as well as their physicians, depend on FDA to ensure that the tests they use to make medical decisions are accurate, reliable, and clinically meaningful.
FDA requirements for many IVDs intended for clinical diagnostic use include premarket review through one of the following pathways:. LDTs are a subset of IVDs that are intended for clinical use and designed, manufactured, and used within a single laboratory. Because such tests historically were relatively simple and available on a limited basis, FDA generally has not enforced premarket review and other legal requirements.
This is because they are developed to diagnose serious or life-threatening diseases or conditions that not only have serious implications for individual patient care, but also for analyses of disease progression and public health decision-making. Thus, FDA requests that developers of such LDTs submit information about their tests to help FDA better understand their design, validation, and performance characteristics. When searching the FDA databases for device products it can be helpful to know the product code of the device or group of devices.
Alternatively, typing the pathogen name, e. The TTFED will in no way involve itself in decision-making that is under the jurisdiction of a member agency. There are two mechanisms, termination or revocation, that result in an EUA test no longer being authorized:. Some circumstances that may result in the revocation of individual EUAs include: reports of adverse events number or severity linked to, or suspected of being caused by, the EUA product; product failure; change in approval status of the EUA product e.
When an EUA product subsequently receives full marketing authorization, then the EUA for that same product and use will be revoked. In addition, since a criterion for the issuance of an EUA is that there is no adequate, approved, and available alternative, when an EUA product subsequently receives full marketing authorization, such product may be an adequate, approved and available alternative to other remaining EUA products. In such cases, FDA anticipates providing notice of intent to revoke to the remaining relevant EUA holders and an invitation to the holder of the full marketing authorization to comment on the extent to which their product is or is not an adequate, approved, and available alternative to existing EUA products.
When an EUA is revoked or terminated, there will be a date beyond which the test s can no longer be used. The EUA holder of the revoked or terminated EUA is advised to instruct laboratories to discontinue use of and discard any remaining EUA product inventory in accordance with the terms of the revocation or termination.
The letter of authorization will prescribe to whom and how to report performance concerns. Deviations from the procedures outlined are not permitted under the EUA. For example, if a laboratory is requested to test specimens that are not authorized with the specific test, then testing the requested specimen would be an illegal use of the test.
When providing test results to healthcare providers and patients, laboratories must include a copy of the authorized fact sheets along with reports of the test results.
Under exigent circumstances, other appropriate methods for disseminating fact sheets may generally be used, as described in the EUA Authorization Letter for a specific test, which may include mass media. Similarly, if a laboratory wishes to see a claim, instrument or specimen, etc. Laboratories are welcome to request an EUA for their specific test, provided all criteria are met.
Defining Qualification, Verification, and Validation
If an EUA is revoked or terminated, then there will be a date beyond which the test can no longer be used for clinical diagnostic purposes. If you have additional questions, contact Device fda.With our ever-increasing understanding of the molecular basis of disease, clinical laboratories are implementing a variety of molecular diagnostic tests to aid in the diagnosis of hereditary disorders, detection and monitoring of cancer, determination of prognosis and guidance for cancer therapy, and detection and monitoring of infectious diseases.
Before introducing any new test into the clinical laboratory, the performance characteristics of the assay must be "verified," if it is a US Food and Drug Administration FDA -approved or FDA-cleared test, or "validated," if it is a laboratory-developed test.
Although guidelines exist for how validation and verification studies may be addressed for molecular assays, the specific details of the approach used by individual laboratories is rarely published.
Many laboratories, especially those introducing new types of molecular assays, would welcome additional guidance, especially in the form of specific examples, on the process of preparing a new molecular assay for clinical use.