Lack of regulations and quality management jeopardizes the quality and safety of LDTs, claim experts in clinical laboratory medicine in a commentary to Canadian policymakers
The IHPME members published their comments in the Canadian Medical Association Journal (CMAJ), a peer-reviewed journal owned by Joule Inc., a subsidiary of the Canadian Medical Association. In it, they claim “recent expansion of the molecular diagnostics industry has revealed weaknesses in Canada’s regulatory system for laboratory-developed tests, which are not subject to statutory regulations on medical devices.”
For pathologists and clinical laboratory professionals in both Canada and the United States, these recent actions show the concerns many experts have as they watch the explosive growth in the use of laboratory-developed tests in both countries. In many ways, the swift advances in molecular and genetic diagnostics is outrunning the ability of government regulators to keep pace with use of LDTs in clinical care settings.
In their commentary in CMAJ, the IHPME members also
claim the review and evaluation of LDTs in Canada is inconsistent. Some LDTs they
say, may endure stringent assessments and have endorsements by clinical
guidelines or findings that are published in scientific journals. Other LDTs,
however, may have no analysis at all.
In addition, the IHPME members point out that there is no
national registry kept of LDTs. They theorize that a lack of proper regulation,
controls, and quality management “has potentially jeopardized the delivery of
quality, safe, timely, and appropriate care.”
The researchers calling on Health Canada to address these
issues include:
Fiona A. Miller, PhD, Professor of Health Policy and IHPME Chair in Health Management Strategies;
François Rousseau, PhD, Professor, Department of Molecular Biology, Medical Biochemistry and Pathology, Faculty of Medicine, Laval University, Quebec;
Alberto Gutierrez, PhD, Partner, NDA Partners LLC, former Director, Office of In Vitro Diagnostics and Radiological Health at the FDA’s Center for Devices and Radiological Health (CDRH);
Stuart Hogarth, PhD, Lecturer in Sociology of Science and Technology, University of Cambridge, Cambridge, UK.
During an exclusive presentation offered by The Dark Report (Dark Daily’s sister publication) in 2015, Alberto Gutierrez, PhD (above), who at that time was Director, Office of In Vitro Diagnostics and Radiological Health at the FDA, said, “LDTs are an area that will be difficult to regulate. There is a broad set of tests. Some of the LDTs are very good. Some of them require a lot of expertise from the pathologists and some of them don’t. Regulating LDTs in a way that makes sense and that does not disrupt what’s going on [in clinical laboratories] is going to be difficult.” (Photo copyright: FDA.)
Canadian Scientists Call on Health Canada to Take the
Lead on Regulating LDTs
In the US, the FDA has been making moves to regulate LDTs since 2010, with much opposition from clinical laboratories and In Vitro Diagnostic (IVD) manufacturers. The FDA describes LDTs as internally designed clinical laboratory tests that are developed, manufactured, and used within a single laboratory. They have not undergone government regulatory review, can be simple or complex, and can be utilized to detect a variety of analytes.
Health Canada is the name of a department that falls under
the purview of the Minister of
Health and is part of Canada’s Health
Portfolio. It is responsible for helping Canadians maintain and improve
their health. Other agencies included in the Health Portfolio are:
According to the IHPME paper, however, Health Canada
currently does not have a way to regulate LDTs, and no government agency in
that country is responsible for the oversight of laboratory-developed tests.
Only LDTs that are marketed as test kits are evaluated and reviewed by Health
Canada.
“The current laboratory regulatory system in Canada involves a mixture of public and private entities and operates with oversight from provincial governments, nongovernmental organizations, and professional societies,” the IHPME paper states, adding, “most provinces and territories rely on voluntary standards that are unevenly applied, with little auditing and systematic testing to ensure quality.”
The authors also note that the current lab regulations in
Canada apply only to the operations of the medical laboratories themselves,
encompassing such things as lab environments, personnel, accreditation, and
quality control. They believe the loophole regarding LDTs needs to be addressed,
and they urged Health Canada to “demonstrate leadership” by subjecting these
tests to regulations that are currently applied to medical devices and
pharmaceuticals.
Other Countries Regulate LDTs, though Not Without
Controversy
In support of their call to action, IHPME researchers noted
that Australia, the EU, and the US all have taken steps to regulate LDTs.
The Australian government began oversight of LDTs in 2010 by
subjecting high-risk LDTs to external evaluation and then tracking them in a
public registry.
An EU regulation, which was passed in 2017, will administer
regulatory review of LDTs manufactured on an industrial scale, which targets
commercial laboratories. The law exempts LDTs utilized within individual
hospital laboratories and should be fully implemented by 2022.
Though on its radar since the 1990s, in 2010, the FDA officially
announced its intent to regulate LDTs in the US. The agency released an initial
draft approach for doing so starting in 2014, held a public workshop on the
topic in 2015, and released a
discussion paper in 2017. At this time, however, the FDA is not regulating
LDTs, though the agency remains open to the possibility.
Dark Daily
has reported extensively over the years on the development of LDTs and the
controversy surrounding the FDA’s moves to regulate them.
According to the FDA
website, problems with several high-risk LDTs have been identified,
including:
Claims that are not adequately supported with
evidence;
Lack of appropriate controls which may yield
erroneous results; and
The FDA’s report, titled, “The
Public Health Evidence for FDA Oversight of Laboratory Developed Tests,” reviewed
20 case studies of LDTs for Lyme disease, ovarian cancer, whooping cough,
fibromyalgia, prostate cancer, autism, breast cancer, melanoma, Vitamin D, and
other conditions. The agency concluded that in many instances “patients have
been demonstrably harmed or may have been harmed by tests that did not meet FDA
requirements.”
Klein noted, however, that “The 20 tests described by FDA are mostly a hodgepodge of outlier assays including tests that were never offered, tests for which comparable FDA assays perform poorly, tests for poorly defined disorders with psychologic components, and use of an FDA-approved test off-label.” He continued, “That FDA could find only these dubious examples out of the many thousands of laboratory-developed procedures (LDPs) that benefit patients each day, calls into question the agency’s rationale for expanding its regulatory scope to include LDPs.”
Perhaps this is why the FDA has yet to implement regulations
for LDTs. The controversy continues.
Whether Health Canada will accept the advice of the IHPME
scientists and take steps to regulate laboratory-developed tests in Canada remains
to be seen. As more LDTs are created and manufactured, however, it is probable
that governments will continue to evaluate the administration and oversight of laboratory-developed
tests.
In both Canada and the United States, pathologists, clinical
laboratory managers, and executives at in vitro diagnostic manufacturers
can expect an ongoing tug-of-war between government regulators and the lab
industry over the most appropriate ways to regulate LDTs.
Liquid biopsy tests hold much promise. But inconsistencies in their findings provoke scrutiny and calls from researchers for further development before they can be considered reliable enough for diagnostic use
Many commercial developers of liquid biopsy tests tout the accuracy and benefits of their diagnostic technology. However, there are an equal number of medical laboratory experts who believe that this technology is not yet reliable enough for clinical use. Critics also point out that these tests are being offered as Laboratory Developed Tests (LDTs), which are internally developed and validated and have not undergone regulatory review.
Dark Daily has published several e-briefings on researchers who have sent the same patient samples to different genetic testing labs and received back materially different test results. Now, a new study by Johns Hopkins University concludes that liquid biopsy technology “must improve” before it should be relied upon for diagnostic and treatment decision making.
‘Certification for Medical Laboratories Must Improve’
Liquid Biopsy is the term for drawing whole blood and looking for cancer/tumor cells circulating in the blood stream. This is one factor in the imprecision of a liquid biopsy. Did the blood sample drawn actually have tumor cells? After all, only a limited number of tumor cells, if present, are in circulation.
Gonzalo Torga, MD (above left), and Kenneth J. Pienta, MD (above right), are the two Johns Hopkins Medicine doctors who conducted the recent study into the efficacy of liquid biopsy laboratory developed tests (LDTs) offered by different medical laboratory companies. They published their findings in JAMA Oncology. (Photos copyright: Johns Hopkins.)
In reporting the DNA findings and results from the two medical laboratory companies, researchers discovered that the results completely matched in only three of the 40 patients! The Johns Hopkins researchers are concerned that patients could be prescribed certain cancer treatments based on which lab company’s liquid biopsy test their physician orders, instead of an accurate identification of the unique mutations in their tumors.
“Liquid biopsy is a promising technology, with an exceptional potential to impact our ability to treat patients, but it is a new technology that may need more time and experience to improve,” Gonzalo Torga, MD, Postdoctoral Fellow and Instructor at Johns Hopkins, and the lead author of the study, told Forbes. “We can’t tell from these studies which laboratory’s panel is better, but we can say that certification for these laboratories must improve.”
Unlocking New View of Tumors
Two commercial tests were used for the study:
Guardant360 from Guardant Health, Inc., uses digital sequencing to analyze genomic data points at the single molecular level. It examines 73 genes, including all National Comprehensive Cancer Network (NCCN) listed genes. The test searches for DNA fragments among billions of cells and digitally tags each fragment. This process unlocks a view of tumors that is not seen with tissue biopsies, which helps doctors prescribe the best treatment options for a particular patient.
“As a simple blood test, it provides physicians with a streamlined, cost-effective method to identify genomic alterations that can comprehensively influence a patient’s therapy response,” Helmy Eltoukhy, PhD, co-founder and Chief Executive Officer at Guardant Health, told MDBR.
“The only way of keeping ahead of those diseases and tracking those mutations has been through surgery, through doing a tissue biopsy and physically cutting a piece of the tumor out and sequencing it,” Eltoukhy noted in an interview with Xconomy. “What we’re able to do is essentially get the same, or sometimes better performance to tissue biopsy, but through two teaspoons of blood.”
According to the Guardant Health website, it takes just 14 days for a full report from Guardant360 to reach the ordering physician. In addition, the blood test provides samples with an adequate level of cell-free DNA to test 99.8% of the time and reduces errors and false positives found in standard sequencing methods by 1,000 times. It is common for samples used for tissue sequencing to have insufficient DNA for testing 20% to 40% of the time.
“We believe that conquering cancer is at its core a big data problem, and researchers have been data-starved,” explained Eltoukhy in VentureBeat. “Our launch of the world’s first commercial comprehensive liquid biopsy sparked a boom in cancer data acquisition. Every physician who orders one of our tests, and every patient whose tumor DNA we sequence, adds to this larger mission by improving our understanding of this complex disease.”
PlasmaSELECT-R64, manufactured by Personal Genome Diagnostics (PGDx), evaluates a targeted panel of 64 genes that have biological and functional relevance in making treatment decisions. PGDx announced the expanded version of its PlasmaSELECT assay in March of 2017.
“We are proud to launch the revolutionary PlasmaSELECT 64 expanded assay just six months after we introduced the most accurate, clinically actionable liquid biopsy tumor profiling assay to the market,” said Doug Ward, Chief Executive Officer at PGDx, in a press release. “This update is the first liquid biopsy assay that includes MSI (microsatellite instability) testing as a biomarker for high tumor mutational load, thereby providing cancer patients and their oncologists with information on whether they might be candidates for immuno-oncology therapies. The ability to generate DNA tumor profiling non-invasively using blood or plasma offers many advantages and makes genomic testing more accessible and usable.”
Regulations of LDTs Could be Needed to Improve Liquid Biopsy Tests
There are pathologists and clinical laboratory professionals who believe the technology behind liquid biopsies is not yet reliable enough for clinical use. The tests are being offered as LDTs, which are internally developed and validated, and the Food and Drug Administration (FDA) allows LDTs to be sold without regulatory reviews at this time. However, there are discussions regarding if and how to regulate LDTs, the outcome of which could impact how clinical laboratories are allowed to market the LDTs they develop.
Clearly, liquid biopsies are still in their relatively early stages of development. More testing and evaluation is needed to determine their efficacy. However, their potential to revolutionize cancer detection and care is obvious and a strong motivator for LTD developers, which means there will be future developments worth noting.
In recent weeks, company representatives began informing physicians at cancer centers and hospitals about a problem with a specific variant in the MSH2 gene
Invitae Corporation (NYSE:NVTA), a genetic testing company in San Francisco, has told some physicians and clinicians in recent weeks that it has reported false-negative results. Clinical laboratory professionals with knowledge of the facts in this case believe the cause of the false negative results may have gone undetected for months and that genetic tests for a large number of patients may be involved.
For several weeks, Invitae has reported to its ordering physicians that it knows about a small number of false-negative reports that affect an estimated two to 10 patients who have a rare genetic variant in the MSH2 gene. The variant is known as the Boland Inversion and the gene is associated with Lynch syndrome (AKA, hereditary nonpolyposis colorectal cancer).
This episode may be a watershed event in the evolution of the genetic testing industry. Evidence indicates that genetic tests for a large number of patients were done incorrectly, and that the problem was systemic and went undetected by the lab company’s staff for as long as 11 months. Because these genetic tests were laboratory-developed tests (LDTs), the problem at Invitae could be used by some to argue that FDA regulation of LDTs is needed.
Invitae provided two written statements to The Dark Report, Dark Daily’s sister publication. The full statements can be read at the end of this story. The Dark Report is preparing a detailed intelligence briefing about this matter in its upcoming August 28 issue.
False Negative Reports for Some Genetic Tests
In one statement, the company wrote, “For the past several weeks Invitae has been working with clinicians to address an issue related to our analysis of a rare genetic variant in the MSH2 gene associated with Lynch syndrome (0.007% of inherited cancer tests), also known as the Boland inversion, which we believe could have led to a false negative report for a small number of patients (estimated 2-10 patients impacted).”
Invitae Corporation, founded in 2010, is a clinical laboratory company based in San Francisco that provides genetic testing services and has used aggressive pricing to fuel fast growth in specimen volume in recent years. According to the company’s 2017 second quarter earnings report, for the first six months of 2017 Invitae reported revenue of $27.7 million. Its net loss before taxes for the first six months of 2017 was $57.3 million. (Photo copyright: Yelp.)
After detecting the problem, the company began a root-cause analysis to determine the extent of the problem. “We would expect to observe the MSH2 Boland inversion event in 0.007% of patients undergoing hereditary cancer testing and approximately one in every 1,250 in patients with Lynch syndrome-spectrum cancers,” the company stated. “Based on these estimates, we expect this to impact a very small number of patients.”
Limited Number of Patients with False Negatives, but How Many Tests Involved?
The number of patients whose test results may have been affected is the subject of speculation among medical laboratory professionals who refer genetic tests to Invitae. Two medical directors at genetic testing laboratories pointed out that—based on the lab company’s estimate that false negatives were reported on just two to 10 patients—the problem could involve 3,000 to 12,500 patients.
The photo above taken Feb. 12, 2015, is of the Invitae management team at a happier time when the company’s shares began trading on the New York Stock Exchange (NYSE). (Photo copyright: Invitae.)
The company’s internal quality systems did not identify this problem. They learned about the problem from an outside source. Invitae said, “A client recently reported a discrepancy between an Invitae report and a report issued by another laboratory for the presence of a single rare mutation in MSH2, known as the Boland inversion. As soon as we learned of the discrepancy, we quickly identified and rectified the issue.”
The company confirmed that the problem with the Boland Inversion had gone undiscovered for 11 months, stating “We have identified all samples impacted by this issue, which were processed between September 2016 and July 2017. We are reanalyzing all previous results over the next several weeks to ensure their accuracy.”
Quality Control Checks for Omission of Assay Components
In its statement, Invitae referenced the quality-control issue, saying, “Because of the unique characteristics of how we were testing for the MSH2 Boland inversion, our quality control checks did not catch omission of the components of the assay. As soon as the omission was recognized and relevant components returned to the assay, it once again performed properly. We have added two separate quality controls to ensure this issue will not reoccur.”
The statement continued, saying, “We have identified all samples that could have been impacted by this error and are in the process of reanalyzing them free-of-charge.” The company also said, “Our ability to detect this specific MSH2 mutation has been fully restored and is functioning properly …. Moving forward, the new assay incorporates a quality check for successful capture and sequencing of the region around both ends of the Boland inversion so that the absence of the inversion gives a positive signal and the presence of the inversion gives a separate and different signal, while failure or absence of successful capture of these regions gives a third and different signal.”
To comply with federal and state clinical laboratory regulations, Invitae confirmed that it had conducted a root cause analysis (RCA) and was addressing the problem, notifying physicians and patients as necessary.
“We have been reaching out to all customers with patients who could have been impacted by this issue. We have samples to conduct reanalysis for all patients and will reach out to individual clinicians if any of those samples are deemed ‘quantity not sufficient’ (QNS) and new ones are required. However, the assay developed and validated for reanalysis is designed to use very small amounts of DNA, so we anticipate the number of new samples needed will be small.”
Retesting may be a greater challenge for Invitae than is indicated by its statement. Several pathologists told The Dark Report that such retesting comes with several problems. For example, certain states require patient samples used in genetic testing to be destroyed within a set time period. In such cases, the lab would need to work with the physician to have the patient provide a new sample for the retest. Also, it is common for genetic testing to use so much of a sample that the amount remaining is inadequate for a retest. In these cases, a new sample must be collected.
Another issue for Invitae will involve both the time to retest and the cost to retest. One lab executive pointed out that Invitae’s lab accessions had almost tripled from 12,500 in the second quarter of 2016 to 30,500 in second quarter of 2017. “Their lab is already straining to stay up with that volume increase. If Invitae must retest, say, 10,000 or more patient samples because of the MSH2 Boland inversion problem, this can seriously overload their lab and cause significant delays in turnaround time for all samples,” he explained.
A point of interest for pathologists and laboratory directors is whether any clinicians have filed a complaint or notified Invitae’s laboratory accreditation organizations, and the federal and state lab regulators, about the problems they had with this specific MSH2 mutation in their genetic tests and genetic test panels. Invitae has not commented on that situation.
Finally, the consequences for the patients whose genetic tests were performed by Invitae during this 11-month period should be considered. One executive from a large genetic testing lab in the Northeast said it best: “Every lab that performs genetic testing is in the rare mutation business!” he declared. “The mission is specifically to test for rare mutations and accurately identify the 1-in-1,000,000 mutation to enable that patient to get the right treatment that is invariably life-changing.
“Thus, for any lab like Invitae to tell its physicians that ‘only a few patients’ may have been given a false negative result from their genetic test betrays the quality and accuracy that all physicians, patients, and their families expect of our labs,” he continued. “Remember that what physicians and patients do with these results is very drastic! I consider it a massive failure anytime a genetic lab—whether large or small—misses rare mutations in even a small number of patients because of problems at the bench.”
—The Dark Report Editorial Team
Invitae provided two statements to The Dark Report. Here is the first statement, in full, dated Aug. 24 from Invitae, titled, “Statement on Boland Inversion Testing.”
“For the past several weeks Invitae has been working with clinicians to address an issue related to our analysis of a rare genetic variant in the MSH2 gene associated with Lynch syndrome (0.007% of inherited cancer tests), also known as the Boland inversion, which we believe could have led to a false negative report for a small number of patients (estimated 2-10 patients impacted).
“Our immediate priority has been getting accurate and actionable information to patients and clinicians about what happened and the steps we are taking to address the situation. We have identified all samples that could have been impacted by this error and are in the process of reanalyzing them free-of-charge. We have been personally reaching out to clinicians whose patients may have been impacted to discuss the issue, outlining what we have done to correct it and explaining the timeframe for receiving updated information. We are also offering no-cost family variant testing for the immediate families of any patient who tests positive for the Boland inversion variant, something we do for all our patients who test positive for a pathogenic variant.
“Our ability to detect this specific MSH2 mutation has been fully restored and is functioning properly. The corrected assay has been revalidated and shown to have regained its previous high sensitivity for the Boland inversion for all samples currently in the lab. Dual quality control checks specific to this issue have been added and are performing properly. We have also reviewed all of our other testing and are confident that this was an isolated error. Our ability to detect other MSH2 mutations or mutations in any other genes in our testing panels was not affected.
“Because of the unique characteristics of how we were testing for the MSH2 Boland inversion, our quality control checks did not catch omission of the components of the assay. As soon as the omission was recognized and relevant components returned to the assay, it once again performed properly. We have added two separate quality controls to ensure this issue will not reoccur.
“We take the reliability and validity of our test results extremely seriously. Nothing is more important to our company than ensuring that the information we provide is accurate. It is extremely rare that we find an error, but when we do we will quickly to correct it and share information with clinicians, in keeping with the medical community’s standards for error reporting.”
This is the second written statement, in full, provided to The Dark Report on Aug. 24. Invitae said:
“A client recently reported a discrepancy between an Invitae report and a report issued by another laboratory for the presence of a single rare mutation in MSH2, known as the Boland inversion. As soon as we learned of the discrepancy, we quickly identified and rectified the issue.
“We have identified all samples impacted by this issue, which were processed between September 2016 and July 2017. We are reanalyzing all previous results over the next several weeks to ensure their accuracy.
“We would expect to observe the MSH2 Boland inversion event in 0.007% of patients undergoing hereditary cancer testing, and approximately one in every 1,250 in patients with Lynch syndrome-spectrum cancers. Based on these estimates, we expect this to impact a very small number of patients.
“Moving forward, the new assay incorporates a quality check for successful capture and sequencing of the region around both ends of the Boland inversion so that the absence of the inversion gives a positive signal and the presence of the inversion gives a separate and different signal, while failure or absence of successful capture of these regions gives a third and different signal.
“We have been reaching out to all customers with patients who could have been impacted by this issue. We have samples to conduct reanalysis for all patients and will reach out to individual clinicians if any of those samples are deemed ‘quantity not sufficient’ (QNS) and new ones are required. However, the assay developed and validated for reanalysis is designed to use very small amounts of DNA, so we anticipate the number of new samples needed will be small.”
Report states IVD companies are focusing on core lab, seeking China FDA approval, and targeting urgent care
Several of the same powerful trends reshaping healthcare and clinical laboratory services are having equally significant influence on in vitro diagnostics (IVD) manufacturers. In particular, the consolidation of hospitals and physicians, as well as the emergence of new sites of service—such as urgent care centers and retail clinics—are motivating IVD companies to tailor new diagnostic systems to the unique needs of these entities.
Kalorama, a division of MarketResearch.com, has released its list of Top-Trends that will affect IVD developers in 2017. IVDs are at the heart of the medical laboratory industry. Thus, these reports are critical to keeping clinical laboratory managers and pathology groups informed on anything that could affect the production, voracity, and availability of diagnostic testing. (more…)
Researchers suggest FDA approvals be based on more rigorous clinical studies before and after medical devices get approved to market
Do regulators consistently get it right when reviewing medical devices and clinical laboratory tests as part of the market-approval process? New findings provide credible evidence that government regulators have their own problems when reviewing submissions for market clearance.
Essentially, if these researchers can find obvious flaws in how the FDA reviews a relatively limited number of medical devices, as identified by this study, then how would the FDA cope with the need to review thousands of LDTs in a timely, cost-effective, and rigorous manner? (more…)
