Working from tissue slides similar to those used by surgical pathologists, the algorithm accurately detects prostate cancer with an impressive 98% sensitivity
It could be that a new milestone has been reached on the road to using artificial intelligence (AI) to help anatomic pathologists diagnose cancer and other diseases. A research collaboration between a major American university and an Israeli company recently published a study about the ability of an AI algorithm to correctly diagnose prostate cancer.
The scientists trained the Galen Prostate AI to recognize prostate cancer by having it examine images from over a million parts of stained tissue slides taken from patient biopsies. Expert pathologists labeled each image to teach the algorithm how to distinguish between healthy and abnormal tissue. The AI was then tested on 1,600 different tissue slide images that had been collected from 100 patients seen at UPMC who were suspected of having prostate cancer.
“Humans are good at recognizing anomalies, but they have their own biases or past experience,” said Rajiv Dhir, MD, Chief Pathologist and Vice Chair of Pathology at UPMC Shadyside Hospital, Professor of Biomedical Informatics at University of Pittsburgh, and senior author of the study, in a UPMC news release. “Machines are detached from the whole story. There’s definitely an element of standardizing care.”
The image above is “of prostate cancer (represented by the heatmap) detected by the Ibex Galen Prostate [AI] solution on a biopsy that was previously diagnosed as benign by the pathologist,” stated an Ibex news release announcing the UPMC study. (Photo copyright: Ibex.)
UPMC Algorithm Goes Beyond Cancer Detection, Exceeds Human Pathologists
The researchers also noted that this is the first algorithm to extend beyond cancer detection. It reported high performance for tumor grading, sizing, and invasion of surrounding nerves—clinically important features of pathology reports.
“Algorithms like this are especially useful in lesions that are atypical,” Dhir said. “A nonspecialized person may not be able to make the correct assessment. That’s a major advantage of this kind of system.”
The algorithm also flagged six slides as potentially containing abnormal tissue that were not flagged by human pathologists. However, the researchers pointed out that this difference does not mean the AI is better than humans at detecting prostate cancer. It is probable, for example, that the pathologists simply saw enough evidence of malignancy elsewhere in the patients’ samples to recommend treatment.
Other Studies Where AI Detected Prostate Cancer
The UPMC researchers are not the first to use AI to detect prostate cancer. In February, The Lancet Oncology published a study from researchers at Radboud University Medical Center (RUMC) in the Netherlands who developed a deep learning AI system that could determine the aggressiveness of prostate cancer in certain patients.
For that research, the RUMC scientists collected 6,000 biopsies from more than 1,200 men. They then showed the biopsy images along with the original pathology reports to their AI system. Using deep learning, the AI was able to detect and grade prostate cancer according to the Gleason Grading System (aka, Gleason Score), which is used to rate prostate cancer and choose appropriate treatment options. The Gleason Score ranges from one to five and most cancers obtain a score of three or higher.
“Systems such as ours can be used in different ways. First, it can be used to screen biopsies and to filter out the easy (benign) cases. This could reduce the workload for pathologists,” said Wouter Bulten, a PhD candidate at Radboud who worked on the study, in an interview with HemOnc Today. “Second, the system can be used as a second opinion after the pathologist’s initial read. The system can flag a case if its opinion differs from that of the pathologist. It also can give feedback during the first read, showing the pathologist where to look. In this case, the pathologist needs only to confirm the opinion of the AI system.”
Can Today’s AI Outperform Human Pathologists?
In their research, the Radboud team discovered that their AI system was able to achieve pathologist-level performance and, in some cases, even performed better than human pathologists. However, they do not foresee AI replacing the need for pathologists, but rather emerging as another method to use in cancer detection and treatment.
“We see our system as an additional tool that the pathologist can use. Although our system performs very well, it still makes mistakes,” stated Bulten. “These mistakes are often different from those a human would make. We believe that when you merge the expertise of the pathologist with the second opinion of an AI system, you get the best of both worlds.”
According to the American Cancer Society, prostate cancer is the second most common cancer among men in the US, after skin cancer. The organization estimates there will be approximately 191,930 new cases of prostate cancer diagnosed and about 33,330 deaths from the disease in the US in 2020.
Though the UPMC study focused only on prostate cancer, the scientists believe their algorithm can be trained to detect other types of cancer as well. AI in clinical diagnostics is clearly progressing, however more studies will be required. Nevertheless, if AI can truly become a useful tool for anatomic pathologists to detect cancer earlier, we may see a welcomed reduction in cancer deaths.
Clinical laboratories involved in genetic testing may find this welcomed news, after a pair of studies conducted in 2019 raised concerns about CRISPR base editing. The researchers of those studies observed that it “causes a high number of unpredictable mutations in mouse embryos and rice,” Chemical and Engineering News (C&EN) reported, adding, “Other groups have raised concerns about off-target mutations caused when the traditional CRISPR-Cas9 form of gene editing cuts DNA at a location that it wasn’t supposed to touch. The results of the new studies are surprising, however, because scientists have lauded base editors as one of the most precise forms of gene editing yet.”
Nevertheless, UC Berkeley’s latest breakthrough is expected to drive development of new and more accurate CRISPR-Cas genome-editing tools, which consist of base editors as well as nucleases, transposases, recombinases, and prime editors.
Understanding CRISPR Base Editors At a ‘Deeper Level’
Harvard University Chemistry and Chemical Biology Professor David Liu, PhD, who co-authored the study, explained the significance of this latest discovery.
“While base editors are now widely used to introduce precise changes in organisms ranging from bacteria to plants to primates, no one has previously observed the three-dimensional molecular structure of a base editor,” he said in a UC Berkeley news release. “This collaborative project reveals the beautiful molecular structure of a state-of-the-art highly-active base editor—ABE8e—caught in the act of engaging a target DNA site.”
UC Berkeley Professor Jennifer Doudna, PhD (above), who served as senior author of the study, says scientists may now have the information necessary to refine base editors and improve their precision and genome-targeting ability. “This structure helps us understand base editors at a much deeper level,” she said in the UC Berkeley statement. “Now that we can see what we’re working with, we can develop informed strategies to improve the system.” (Photo copyright: UC Berkeley.)
Jennifer Doudna, PhD, UC Berkeley Professor, Howard Hughes Medical Institute Investigator, and senior author of the study, has been a leading figure in the development of CRISPR-Cas9 gene editing. In 2012, Doudna and Emmanuelle Charpentier, PhD, Founding, Scientific and Managing Director at Max Planck Unit for the Science of Pathogens in Berlin, led a team of researchers who “determined how a bacterial immune system known as CRISPR-Cas9 is able to cut DNA, and then engineered CRISPR-Cas9 to be used as a powerful gene editing technology.” In a 2017 news release, UC Berkeley noted that the work has been described as the “scientific breakthrough of the century.”
Viewing the Base Editor’s 3D Shape
CRISPR-Cas9 gene editing allows scientists to permanently edit the genetic information of any organism, including human cells, and has been used in agriculture as well as medicine. A base editor is a tool that manipulates a gene by binding to DNA and replacing one nucleotide with another.
According to the recent UC Berkeley news release, the research team used a “high-powered imaging technique called cryo-electron microscopy” to reveal the base editor’s 3D shape.
Genetic Engineering and Biotechnology News notes that, “The high-resolution structure is of ABE8e bound to DNA, in which the target adenine is replaced with an analog designed to trap the catalytic conformation. The structure, together with kinetic data comparing ABE8e to earlier ABEs [adenine base editors], explains how ABE8e edits DNA bases and could inform future base-editor design.”
The graphic above, taken from the UC Berkeley news release, shows the “3D structure of a base editor, comprised of the Cas9 protein (white and gray), which binds to a DNA target (teal and blue helix) complementary to the RNA guide (purple), and the deaminase proteins (red and pink), which switch out one nucleotide for another.” (Image and caption copyright: UC Berkeley.)
Knowing the Cas9 fusion protein’s 3D structure may help eliminate unintended off-target effects on RNA, extending beyond the targeted DNA. However, until now, scientists have been hampered by their inability to understand the base editor’s structure.
“If you really want to design truly specific fusion protein, you have to find a way to make the catalytic domain more a part of Cas9, so that it would sense when Cas9 is on the correct target and only then get activated, instead of being active all the time,” study co-first author Audrone Lapinaite, PhD, said in the news release. At the time of the study, Lapinaite was a postdoctoral fellow at UC Berkeley. She is now an assistant professor at Arizona State University.
“As a structural biologist, I really want to look at a molecule and think about ways to rationally improve it. This structure and accompanying biochemistry really give us that power,” added UC Berkeley postdoctoral fellow Gavin Knott, PhD, another study co-author. “We can now make rational predications for how this system will behave in a cell, because we can see it and predict how it’s going to break or predict ways to make it better.”
Clinical laboratory leaders and pathologists will want to monitor these new advances in CRISPR technology. Each breakthrough has the power to fuel development of cost-effective, rapid point-of-care diagnostics.
Pooled testing could become a critical tool for clinical laboratories to spot the SARS-CoV-2 coronavirus among asymptomatic and pre-symptomatic individuals
COVID-19 testing for individuals has expanded in the US, but the number of people actually tested remains a small proportion of the country’s total population and clinical laboratory testing supply shortages continue to hamper progress. A technique known as pooled testing may help. Federal experts hope it will substantially increase the number of individuals who are tested for the SARS-CoV-2 coronavirus before it makes a possible resurgence in the fall.
One-by-one, some of the nation’s largest clinical laboratory organizations are developing the capability to do pooled testing. For example, on July 18, the Food and Drug Administration (FDA) announced it had issued Quest Diagnostics (NYSE:DGX) an Emergency Use Authorization (EUA) for its SARS-CoV-2 rRT-PCR test, and that it is valid for up to four individual samples as a pooled test.
Quest’s rRT-PCR test was the first COVID-19 diagnostic test to be authorized for use with pooled samples, the FDA noted in a new release.
In the FDA’s statement announcing Quest’s EUA for its rRT-PCR test, Stephen M. Hahn, MD (above), FDA Commissioner, said, “This EUA for sample pooling is an important step forward in getting more COVID-19 tests to more Americans more quickly while preserving testing supplies.” He added, “Sample pooling becomes especially important as infection rates decline and we begin testing larger portions of the population.” (Photo copyright: CBS News.)
Following the announcement of Quest’s EUA, on July 24 the FDA announced LabCorp’s (NYSE:LH) EUA for its COVID-19 real-time reverse transcription polymerase chain reaction (rRT-PCR) test. The test, the EUA states, is intended for the “qualitative detection of nucleic acid from SARS-CoV-2 in upper and lower respiratory specimens” in individuals suspected of COVID-19, using “a matrix pooling strategy (i.e., group pooling strategy), containing up to five individual upper respiratory swab specimens (nasopharyngeal, mid-turbinate, anterior nares or oropharyngeal swabs) per pool and 25 specimens per matrix.”
Exponentially Increasing Testing
In pooled testing, instead of performing a coronavirus test on every specimen received by a clinical laboratory, samples from each individual specimen are taken and then combined with samples from other specimens. A single test is then performed on the entire collection of specimen samples.
If the results of the pooled samples are negative for coronavirus, it is safe to assume that all the specimens in the batch are negative for the virus. If the pooled sample comes back positive, then it will be necessary to go back to the original specimens in that pooled sample and test each specimen individually.
In an exclusive interview with Dark Daily’s sister print publication The Dark Report, Steven H. Hinrichs, MD, Chair of the Department of Pathology and Microbiology at the University of Nebraska Medical Center (UNMC), noted that one pitfall of pooled testing is that it works best in areas of low virus prevalence.
“For pooled testing, the ideal level of low prevalence would be an infection rate below 10%,” he said, adding, “For COVID-19 test manufacturers, pooled testing has the potential to reduce the number of standard tests labs run by roughly 40% to 60%, depending on the population being tested.
“Cutting the number of COVID-19 tests would be a disadvantage for test manufacturers, because pooled tests would identify large numbers of uninfected individuals who would not require standard testing with EUA tests.
“On the other hand, this policy would be a significant advantage for US labs because pooled testing would cut the number of standard tests,” he continued. “Clinical labs would save money on tests, reagents, and other supplies. It would also ease the burden on the lab’s technical staff,” Hinrichs concluded.
“In our study, we show that it’s reasonable to pool five samples, although we realized that some people may want to pool 10 samples at once,” noted Hinrichs. “But even if one sample is positive in a pool of five, then testing five samples at once saves 80% of our costs if all of those samples are negative. But, if one sample is positive, each of those five samples needs to be retested using the standard test,” Hinrichs explained.
During an American Society for Microbiology (ASM) virtual conference, Deborah Birx, MD, White House Coronavirus Response Coordinator, said, “Pooling would give us the capacity to go from a half a million tests per day to potentially five million individuals tested per day,” STAT reported.
Advantages of using pooled testing for the coronavirus include:
Expanding the number of individuals tested,
Stretching laboratory supplies, and
Reducing the costs associated with testing.
Health officials believe that individuals who have COVID-19 and are asymptomatic are largely responsible for the rising number of coronavirus cases in the US, STAT reported.
“It allows you to test more frequently in a population that may have a low prevalence of disease,” Benjamin Pinsky, MD, PhD, Associate Professor, Departments of Pathology and Medicine at Stanford University School of Medicine, told STAT. “That would allow you to test a lot of negatives, but also identify individuals who are then infected, before they develop symptoms.”
Pooled testing also could be advantageous for communities where COVID-19 is not prevalent, in neighborhoods that need to be tested during an outbreak, and for schools, universities, organizations, and businesses that want to remain safely open while periodically monitoring individuals for the virus, CNN reported.
“The goal is to increase the capacity of testing in a relatively straightforward fashion,” Pinsky told STAT. “The caveat is that by pooling the sample, you’re going to reduce the sensitivity of the test.”
According to Pinsky, “pooling only makes sense in places with low rates of COVID-19, where you expect the large majority of tests to be negative. Otherwise, too many of the pools would come back positive for it to work as a useful surveillance tool,” STAT reported.
As Clinical Lab Testing Increases, Pooled Testing for COVID-19 Could Be Critical
Pooled testing has been used in other countries, including China, to test larger amounts of people for COVID-19.
“If you look around the globe, the way people are doing a million tests or 10 million tests is they’re doing pooling,” Birx said during the ASM virtual conference, CNN reported.
In a press release, the American Clinical Laboratory Association (ACLA) stated that about 300,000 tests for COVID-19 were performed per day in labs across the US in late June. That number was up from approximately 100,000 tests being performed daily in early April.
“All across the country, clinical laboratories are increasing the number of labs processing tests, purchasing additional testing platforms, and expanding the number of suppliers to provide critical testing materials,” said Julie Khani, ACLA President in the press release. “However, the reality of this ongoing global pandemic is that testing supplies are limited. Every country across the globe is in need of essential testing supplies, like pipettes and reagents, and that demand is likely to increase in the coming months.”
Clinical laboratory managers will want to keep an eye on these developments. As the need for COVID-19 testing increases, pooled testing may provide an efficient, cost-effective way to spot the coronavirus, especially among those who are asymptomatic or pre-symptomatic and who display no symptoms.
Pooled testing could become a critical tool in the diagnosis of COVID-19 and potentially decrease the overall number of deaths.
Understanding how superspreading occurs can help clinical lab leaders slow and even prevent the spread of SARS-CoV-2 within their communities and health systems
Clinical laboratories understand the critical importance of preventing the spread of infection. However, according to the Boston Globe, researchers worldwide are learning that roughly 80% of new COVID-19 cases are caused by just 10% of infected people. Those people are called superspreaders.
It’s critical that medical laboratory managers are aware of the role superspreaders play in transmitting SARS-CoV-2, the coronavirus that causes the COVID-19 illness.
Clinical lab leaders who understand how superspreading occurs can take steps to protect staff, patients, and anyone who visits the facility. Because lab personnel such as couriers and phlebotomists, among others, come into contact with large numbers of people daily, understanding how to identify superspreaders could limit transmissions of the coronavirus within the laboratory, as well as within hospital networks.
Superspreading versus Plodding
Influenza and other viruses tend to spread in a way that epidemiologists call “plodding.” One person infects another, and the virus slowly spreads throughout the population. However, scientists around the globe are finding that SARS-CoV-2 transmission does not fit that pattern. Instead, a few infected people appear to be transmitting the virus to dozens of other people in superspreading events, Boston Globe reported.
“You can think about throwing a match at kindling. You throw one match, it might not light the kindling. You throw another match, it may not light the kindling. But then one match hits the right spot and all of a sudden the fire goes up,” Ben Althouse, PhD, principal scientist and co-chair of epidemiology at the Institute for Disease Modeling in Bellevue, Wash., told the Boston Globe.
But because roughly 90% of infected people aren’t spreading the virus, identifying who the superspreaders are can be a challenge. Nevertheless, limiting situations in which superspreading is likely to occur could greatly reduce the spread of infection.
Samuel Scarpino, PhD (above), Assistant Professor in the Network Science Institute at Northeastern University, says that “preventing superspreader events could go a long way toward stopping COVID-19,” Scientific American reported. “All of the data I’m seeing so far suggest that if you tamp down the superspreader events, the growth rate of the infections stops very, very quickly,” Scarpino said. (Photo copyright: University of Vermont.)
Examples of Superspreading Events
One of the first big outbreaks in the United States was an example of a superspreading event. The Biogen (NASDAQ:BIIB) leadership conference in late February in Boston resulted in at least 99 cases of COVID-19 just in Massachusetts, reported the Boston Globe.
Several superspreading events have occurred in houses of worship. One well-documented example prompted a CDC Morbidity and Mortality Weekly Report, titled, “High SARS-CoV-2 Attack Rate Following Exposure at a Choir Practice—Skagit County, Washington.” The 122-member choir met for practice twice in March. On March 3 no one had symptoms, but one person had cold-like symptoms at the March 10 practice. Eventually, 53 members tested positive for SARS-CoV-2.
On May 30, a Texas family held a birthday party, Medical Xpress reported. Twenty-five people attended the party, which only lasted a few hours. The family followed the state’s guidelines for gatherings, however one of the hosts was infected with the SARS-CoV-2 coronavirus and wasn’t aware of it. Seven attendees contracted it, and those seven spread the virus to an additional 10 family members. A total of 18 members of a single family were infected.
There are commonalities among the documented superspreading events. Most occur indoors, often in poorly ventilated areas. Some activities cause more respiratory droplets to be expelled than others, such as singing. Some respiratory droplets are released simply by breathing, and many more are expelled when a person talks. Talking louder expels even more droplets into the air.
Are Some People More Likely to Spread the Coronavirus than Others?
The fact that so few people are responsible for the majority of transmissions of the virus raises questions. Do some people simply have more virus particles to shed? Is biology a factor?
One factor may be how long the SARS-CoV-2 coronavirus is in the body before symptoms of the COVID-19 illness manifest.
“If people got sick right away after they were infected, they might stay at home in bed, giving them few opportunities to transmit the virus,” noted Scientific American in “How ‘Superspreading’ Events Drive Most COVID-19 Spread.” However, CDC states on its website that “The incubation period for COVID-19 is thought to extend to 14 days, with a median time of 4-5 days from exposure to symptoms onset. One study reported that 97.5% of persons with COVID-19 who develop symptoms will do so within 11.5 days of SARS-CoV-2 infection.”
During that time, infected individuals may transmit the virus to dozens of other people. The CDC estimates that about 40% of transmission occurs in pre-symptomatic people, Scientific American reported.
But it’s not all bad news. The fact that circumstances may be more important than biology might be good news for clinical laboratories. “Knowing that COVID-19 is a superspreading pandemic could be a good thing. It bodes well for control,” Nelson told the Boston Globe.
Clinical laboratory managers are encouraged to follow CDC recommended safety protocols, titled, “Guidance for General Laboratory Safety Practices during the COVID-19 Pandemic.” They include social distancing, setting up one-way paths through lab areas, sanitizing shared surfaces such as counters and benchtops, and implementing flexible leave policies so that sick employees can stay home.
Following these guidelines, and being aware of superspreaders, can help medical laboratories and anatomic pathology groups keep staff and customers free of infection.
Clinical laboratories are advised to continue developing methods for making prices for procedures available to the general public
Even as an effective treatment for COVID-19 continues to elude federal healthcare agencies, Medicare officials are pressing ahead with efforts to bring about transparency in hospital healthcare pricing, including clinical laboratory procedures and prescription drugs costs.
In FY 2021 Proposed Rule CMS-1735-P, titled, “Medicare Program; Hospital Inpatient Prospective Payment Systems for Acute Care Hospitals and the Long-Term Care Hospital Prospective Payment System and Proposed Policy Changes and Fiscal Year 2021 Rates; Quality Reporting and Medicare and Medicaid Promoting Interoperability Programs Requirements for Eligible Hospitals and Critical Access Hospitals,” the Centers for Medicare and Medicaid Services (CMS) proposes to “revise the Medicare hospital inpatient prospective payment systems (IPPS) for operating and capital-related costs of acute care hospitals to implement changes arising from our continuing experience with these systems for FY 2021 and to implement certain recent legislation.”
The proposed rule suggests a 1.6% increase (about $2 billion) in reimbursement for hospital inpatient services for 2021, but also eludes to the possibility of payer negotiated rates being used to determine future payment to hospitals.
In its analysis of the proposed rule, Modern Healthcare noted that CMS is “continuing its price transparency push, to the chagrin of some providers.”
However, the provisions in the proposed rule do, according to the CMS news release, advance several presidential executive orders, including:
Controversial Use of Payer Data for Future Medicare Rates
This latest CMS proposed rule (comments period ended July 10) moves forward “controversial price transparency” and has a new element of possible leverage of reported information for future Medicare payment rates, Healthcare Dive reported.
The 1,602-page proposed rule (CMS-1735-P) calls for these requirements in hospital Medicare cost reports:
Median payer-specific negotiated inpatient services;
Inclusion of rates for Medicare Advantage plans and other third party plans;
“In addition, the agency is requesting information regarding the potential use of these data to set relative Medicare payment rates for hospital procedures,” the CMS news release states.
Thus, under the proposed rule, the nation’s 3,200 acute care hospitals and 360 long-term care hospitals would need to start reporting requested data for discharges effective Oct. 1, 2020, a CMS fact sheet explained.
In the news release following the release of the proposed rule, CMS Administrator Seema Verma had a positive spin. “Today’s payment rate announcement focuses on what matters most to help hospitals conduct their business and receive stable and consistent payment.”
However, the American Hospital Association (AHA) articulated a different view, even calling the requirement for hospitals to report private terms “unlawful.”
“We are very disappointed that CMS continues down the unlawful path of requiring hospitals to disclose privately negotiated contract terms,” AHA Executive Vice President Tom Nickels (above) said in a statement, adding, “The disclosure of privately negotiated rates will not further CMS’ goal of paying market rates that reflect the cost of delivering care. These rates take into account any number of unique circumstances between a private payer and a hospital and simply are not relevant for fixing Fee-for-Service Medicare reimbursement.” (Photo copyright: American Hospital Association.)
AHA and other organizations attempted to block a price transparency final rule last year in a lawsuit filed against the U.S. Department of Health and Human Services (HHS), which oversees CMS, Dark Daily reported.
During in-court testimony, provider representatives declared that revealing rates they negotiate with payers violates First Amendment rights, Becker’s Hospital Review reported.
Officials for the federal government pushed back telling the federal judge that they can indeed require hospitals to publish negotiated rates. Hospital chargemasters, they added, don’t tell the full story, since consumers don’t pay those rates, Modern Healthcare reported.
In addition to the increase in inpatient payments and price transparency next steps, the recent CMS proposed rule also includes a new hospital payment category for chimeric antigen receptor (CAR) T-cell therapy. The technique uses a patient’s own genetically-modified immune cells to treat some cancers, as an alternative to chemotherapy and other treatment covered by IPPS, CMS said in the news release.
The agency also expressed intent to remove payment barriers to new antimicrobials approved by the FDA’s Limited Population Pathway for Antibacterial and Antifungal Drugs (LPAD pathway). “The LPAD pathway encourages the development of safe and effective drug products that address unmet needs of patients with serious bacterial and fungal infections,” the CMS fact sheet states.
Clinical laboratories are gateways to healthcare. For hospital lab leaders, the notion of making tests prices easily accessible to patients and consumers will soon no longer be a nice idea—but a legal requirement.
Therefore, clinical laboratory leaders are advised to stay abreast of price transparency regulations and continue to prepare for sharing test prices and information with patients and the general public in ways that fulfill federal requirements.
Though the test initially drew ‘raves’ from Trump administration, the FDA now suggests negative results should be confirmed with an additional ‘high-sensitivity authorized SARS-CoV-2 molecular test’
This spring, as the United States attempted to jump-start a national response to the SARS-CoV-2 coronavirus pandemic, the Trump administration heralded Abbott Laboratories’ five-minute test for COVID-19 as a major breakthrough. But even as the federal Food and Drug Administration (FDA) issued dozens of Emergency Use Authorizations (EUAs) to quickly get COVID-19 diagnostic tests into clinical use, the accuracy of some of those tests came into question—including Abbott’s ID NOW COVID-19 rapid molecular test.
The continuing controversy over Abbott’s ID NOW COVID-19 test shows how the national spotlight can be a double-edged sword, bringing both widespread favorable attention to a breakthrough technology, followed by heightened public scrutiny if deficiencies emerge. At the same time, from the first news stories about the Abbott ID NOW COVID-19 test, pathologists and clinical laboratory managers understood that this test always had certain performance parameters, as is true of every diagnostic test.
“Everybody was raving about it,” a former administration official, speaking on the condition of anonymity to discuss internal deliberations, said of ID NOW in an interview with Kaiser Health News (KHN). “It’s an amazing test, but it has limitations which are now being better understood.”
In a White House ceremony on March 29, 2020, President Trump praised his administration’s role in speeding up development “on both testing and treatment that will help us win our war against the coronavirus.” Among the moves highlighted was the FDA’s approval two days earlier of Abbott’s ID-NOW COVID-19 rapid molecular test (above), which the President stated, “delivers lightning-fast results in as little as five minutes,” adding, “Normally, this approval process from the FDA would take 10 months, and even longer, but we did it in four weeks.” (Photo copyright: Washington Post.)
FDA Warns Public about Inaccurate Test Results
On May 14, the FDA issued a public warning about the point-of-care test’s accuracy after receiving 15 “adverse event reports” indicating some patients were receiving “false negative results.”
“Regardless of method of collection and sample type, Abbott ID NOW COVID-19 had negative results in a third of the samples that tested positive by Cepheid Xpert Xpress when using nasopharyngeal swabs in viral transport media and 45% when using dry nasal swabs,” the NYU study authors stated.
Abbott Rebuts Criticism
In a statement following the FDA’s warning, Abbott said, “We’re seeing studies being conducted to understand the role of ID NOW in ways that it was not designed to be used. In particular, the NYU study results are not consistent with other studies. While we’ve seen a few studies with sensitivity performance percentages in the 80s, we’ve also seen other studies with sensitivity at or above 90%, and one as high as 94%.
“While we understand no test is perfect, test outcomes depend on a number of factors including patient selection, specimen type, collection, handling, storage, transport and conformity to the way the test was designed to be run. ID NOW is intended to be used near the patient with a direct swab test method,” Abbott’s statement added, noting the company would be “further clarifying our product information to provide better guidance” and “reinforcing proper sample collection and handling instructions.”
Then, on May 21, Abbott issued another statement highlighting an interim analysis of an ongoing multisite clinical study demonstrating ID NOW COVID-19 test performance is ≥94.7% in positive agreement (sensitivity) and ≥98.6% negative agreement (specificity) when compared to two different lab-based molecular PCR reference methods.
“We’re pleased ID NOW is delivering on what it was designed to do—quickly detect the virus in people who need to know now if they’re infected,” said Philip Ginsburg, MD, SAIM, Senior Medical Director, Infectious Disease, Rapid Diagnostics at Abbott, in the statement. “This is great news for people who are experiencing symptoms and want to take action before they infect others, reducing the spread of infection in society.”
Nonetheless, KHN reported on June 22 that the FDA had “received a total of 106 reports of adverse events for the Abbott test, a staggering increase. The agency has not received a single adverse event report for any other point-of-care tests meant to diagnose COVID-19.”
Second Comparison Study Results for Abbott’s ID NOW
The Abbott ID NOW test correctly identified 74% of positive samples. In comparison, Cepheid’s Xpert Xpress SARS CoV-2 test correctly identified 99% of positives. Negative agreement was 100% and 92.0% for ID NOW and Xpert, respectively.
The FDA’s testing policy for clinical laboratories and commercial manufacturers recommends diagnostic tests correctly identify at least 95% of positive samples. However, KHN pointed out, a senior FDA official in late May said coronavirus tests that were administered outside lab settings would be considered useful in fighting the pandemic even if they miss 20% of positive cases.
“There’s no way I would be comfortable missing two out of 10 patients,” Whittier told KHN.
Abbott ID-NOW’s Role in the Global Fight to Stop COVID-19
Abbott’s ID NOW COVID-19 test is promoted as delivering positive test results in five minutes and negative results in about 13 minutes. On its website and in news releases, Abbott maintains its test “performs best in patients tested earlier post symptom onset.”
In a July 17 statement, Abbott said, “ We have shipped 5.3 million of our rapid ID NOW tests to all 50 states, Washington DC, Puerto Rico and the Pacific Islands. The majority of these tests have been sent to outbreak hotspots and we’ve asked that customers prioritize frontline healthcare workers and first responders.”
It is common for a new diagnostic instrument and a new clinical laboratory test to be continually improved after initial launch. Thus, the performance of such devices at the time they are given clearance from the FDA to be used in clinical care can be much improved several months or years later.
Given the importance of a reliable point-of-care SARS-CoV-2 test during the pandemic, it can be assumed that Abbott Laboratories is working closely with its medical laboratory customers specifically to improve the accuracy, reliability, and reproducibility of both the instrument and the test kit.
