Incorrect results spanned exams from 2022-2024 with those students affected by this situation stunned by ABPath’s admission
There are always young anatomic pathologists who experience the heartache of failing board exams. Failing, after all, is part of the education process. But how disheartening would it be to learn years later that you actually passed those exams? For about 70 pathology students that’s exactly what happened.
On October 3, the American Board of Pathology (ABPath) announced that “a small percentage” of students were told they had failed an exam when they had not, MedPage Today reported. The test dates ranged from 2022 to 2024 and affected 76 students out of 4,059.
In its official statement, ABPath wrote that it had “recently conducted a reassessment and rescoring of the Primary and Subspecialty certification examinations. In doing so, ABPath identified that a small percentage of candidates (less than 2%) actually met the passing score requirements that were previously scored as slightly below the passing score threshold.”
The organization added, “The impacted candidates have all been notified. Those candidates received a passing score for their respective certification examination and will receive a refund of their application fee, as well as any fees paid toward subsequent examinations. Impacted training programs will be receiving notifications soon.”
According to MedPage Today, which received a copy of the email from a pathology student, “ABPath’s email to affected test takers was brief, explaining that a reassessment of the anatomic pathology certification examination ‘disclosed that your score exceeded the cut-score threshold for a passing score, rather than being slightly below it.’”
Many pathology students were stunned by ABPath’s admission and went to message boards to vent their frustration.
“Failing a major board hits you hard especially after so many years of training and so much time spend on studying. This situation was a shock, a disappointment, and a betrayal,” one former student identified as Idfcwytas wrote on Reddit concerning ABPath’s revelation, MedPage Today reported.
In its coverage of the ABPath announcement, MedPage Today wrote, “Bryan Carmody, MD (above), of Eastern Virginia Medical School in Norfolk, who blogs frequently about medical education issues, said that a situation like this decreases trust in the boards themselves, and that being mistakenly failed has financial and career impacts. For instance, he said, ‘many hospitals require board certification to get hospital privileges or to work there … so if you were applying to an academic job or something, I think it’s going to hurt your application.’” Anatomic pathologists understand clearly how lack of board certification affects their career goals. (Photo copyright: Eastern Virginia Medical School.)
ABPath’s Response and the Impact on Students
An ABPath spokesperson told MedPage Today, “We understand how significant this is to our candidates and diplomats and are committed to working closely with those impacted. ABPath has also implemented improvements to our processes to prevent this issue in the future.”
No details were disclosed on what changes would be made going forward, MedPage Today noted. As a result of the mistake, ABPath refunded exam fees and offered affected testers updated certificates and letters showing the proper results.
Some declared that response does not go far enough. Many claimed ABPath’s efforts lack adequate financial compensation as well as genuine compassion.
Reddit user walleyealx wrote, “The mistake definitely cost me LOTS of time, mental anguish, and money. Please note, there was no apology at all in these emails, which makes me even more angry,” MedPage Today reported.
Several days after the initial email, ABPath offered an apology “for the frustration this has caused,” according to MedPage Today.
No Recourse for Failing Grades
ABPath’s website notes that failed exams cannot be appealed. This means that any students doubting their scores during those years would not have been able to challenge the results.
“Even if the board offers a refund for the exams, it hardly makes up for the time, money, and lost career opportunities these individuals have had to deal with,” noted the Lento Law Firm in a blog post following ABPath’s announcement.
The firm added, “This incident only affected about 2% of exam takers from 2022 to 2024, but it’s a significant issue for these people. Not getting board certification can mean no hospital privileges, which can prevent you from obtaining certain jobs. Failing to get a certain specialty certification can also mean a $30,000 difference in your paycheck. Pathologists have only five years and 10 tries to pass the exam, which costs between $2,100 and $2,600 each time. Re-taking the exam after getting a failing score, therefore, represents a significant financial investment—not to mention the time spent studying.”
Implications of failure can be profound, but mistakes happen. ABPath appears to be taking steps to prevent similar mistakes in the future. Hopefully, the organization will also find a way to make whole those affected by its error.
This may be a new ‘sign of the times’ as hospitals, clinical laboratories, and other healthcare providers working with AI find they also need to hire their own prompt engineers
AI “prompting,” according to Florida State University, “refers to the process of interacting with an AI system by providing specific instructions or queries to achieve a desired outcome.”
According to workable.com, prompt engineers specialize “in developing, refining, and optimizing AI-generated text prompts to ensure they are accurate, engaging, and relevant for various applications. They also collaborate with different teams to improve the prompt generation process and overall AI system performance.”
Healthcare institutions are getting more serious about using AI to improve daily workflows and clinical care, including in the clinical laboratory and pathology departments. But adopting the new technology can be disruptive. To ensure the implementation goes smoothly, hospitals are now seeking prompt engineers to guide the organization’s strategy for using AI.
When Boston Children’s Hospital leaders set out to find such a person, they looked for an individual who had “a clinical background [and] who knows how to use these tools. Someone who had experience coding for large language models and natural language processing, but who could also understand clinical language,” according to MedPage Today.
“We got many, many applications, some really impressive people, but we were looking for a specific set of skills and background,” John Brownstein, PhD, Chief Innovation Officer at Boston Children’s Hospital and Professor of Biomedical Informatics at Harvard Medical School, told MedPage Today.
“It was not easy to find [someone]—a bit of a unicorn-type candidate,” noted Brownstein, who is also a medical contributor to ABC News.
After a four-month search, the hospital hired Dinesh Rai, MD, emergency room physician and AI engineer, for the position. According to Brownstein, Rai had “actually practiced medicine, lived in a clinical environment,” and had “successfully launched many [AI] applications on top of large language models,” MedPage Today reported.
“Some of the nuances I bring to the table in terms of being a physician and having worked clinically and understanding really deeply the clinical workflows and how we can implement the [AI] technology—where its limits are, where it can excel, and the quickest way to get things [done],” Dinesh Rai, MD (above), told MedPage Today. “I’m happy to be able to help with all of that.” Hospital clinical laboratory and pathology managers may soon by engaging with prompt engineers to ensure the smooth use of AI in their departments. (Photo copyright: LinkedIn.)
Prompt Engineers are like F1 Drivers
“It’s kind of like driving a car, where basically anyone can drive an automatic car, and anyone can go onto ChatGPT, write some text, and get a pretty solid response,” said Rai, describing the act of AI prompting to MedPage today.
Then, there are “people who know how to drive manual, and there are people who will know different prompting techniques, like chain-of-thought or zero-shot prompting,” he added. “Then you have those F1 drivers who are very intimate with the mechanics of their car, and how to use it most optimally.”
The American Hospital Association (AHA) believes that AI “holds great promise in helping healthcare providers gain insights and improve health outcomes.” In an article titled, “How AI Is Improving Diagnostics, Decision-Making and Care,” the AHA noted that, “Although many questions remain regarding its safety, regulation, and impact, the use of AI in clinical care is no longer in its infancy and is expected to experience exponential growth in the coming years.
“AI is improving data processing, identifying patterns, and generating insights that otherwise might elude discovery from a physician’s manual effort. The next five years will be critical for hospitals and health systems to build the infrastructure needed to support AI technology, according to the recently released Futurescan 2023,” the AHA wrote.
The graphic above is taken from the American Hospital Association’s article about Futurescan’s 2023 survey results on AI in healthcare. “Healthcare executives from across the nation were asked how likely it is that by 2028 a federal regulatory body will determine that Al for clinical care delivery augmentation (e.g., assisted diagnosis and prescription, personalized medication and care) is safe for use by our hospital or health systems,” AHA stated. This would include the use of AI in clinical laboratories and pathology group practices. (Graphic copyright: American Hospital Association.)
The AHA listed the top three opportunities for AI in clinical care as:
Clinical Decision Tools: “AI algorithms analyze a vast amount of patient data to assist medical professionals in making more informed decisions about care.”
Diagnostic and Imaging: The use of AI “allows healthcare professionals to structure, index, and leverage diagnostic and imaging data for more accurate diagnoses.”
Patient Safety: The use of AI improves decision making and optimizes health outcomes by evaluating patient data. “Systems that incorporate AI can improve error detection, stratify patients, and manage drug delivery.”
The hiring of a prompt engineer by Boston Children’s Hospital is another example of how AI is gaining traction in clinical healthcare. According to the Futurescan 2023 survey, nearly half of hospital CEOs and strategy leaders believe that health systems will have the infrastructure in place by 2028 to successfully utilize AI in clinical decision making.
“I’m lucky to [be] in an organization that has recognized the importance of AI as part of the future practice of medicine,” Rai told MedPage Today.
Pathologists and managers of clinical laboratories and genetic testing companies will want to track further advancements in artificial intelligence. At some point, the capabilities of future generations of AI solutions may encourage labs to hire their own prompt engineers.
Coordinating at-home testing for monkeypox may provide opportunities for clinical laboratories to add value for their physician clients
Microbiologists and clinical laboratory managers who oversee medical laboratory tests for monkeypox (aka, mpox) will be interested to learn that, according to the US Centers of Disease Control (CDC), cases per day have dropped into the single digits.
The United States led the world in cases during the 2022-2023 outbreak, according to the most recent CDC statistics. As of February 15, the US has reported 30,193 cases of monkeypox with 32 deaths.
Nevertheless, January 31 was the day that the US public health emergency involving monkeypox officially expired. Data from the World Health Organization shows the number of daily monkeypox cases in most countries around the world is declining, although numbers of cases are still increasing in some South American countries.
The global monkeypox outbreak appears to have slowed considerably, but are we out of the woods?
“There were concerns that there would be ongoing transmission and that ongoing transmission would become endemic in the United States like other STIs: gonorrhea, chlamydia, syphilis. We have not seen that occur,” Jonathan Mermin, MD (above), Director of the National Center for HIV, Viral Hepatitis, STD, and TB Prevention at the CDC, told CNN.”We are now seeing three to four cases a day in the United States, and it continues to decline. And we see the possibility of getting to zero as real.” This decline in monkeypox test corresponds with a similar decline in COVID-19 clinical laboratory testing as well. (Photo Copyright: CDC.)
Untried Vaccine and At-home Testing for Monkeypox
When the monkeypox outbreak began in May of 2022, there were concerns about the US’ level of preparedness for dealing with a second pandemic while also battling COVID-19. But monkeypox was not entirely unknown to the scientific and medical communities.
Monkeypox first appeared in 1958 amongst a colony of monkeys being kept for research. The origin of the disease is not known. According to the CDC, the first reported human case of monkeypox was in 1970. Prior to the 2022 outbreak, most cases were found in central and western African countries. Cases outside of those areas could be traced back to travel from those specific countries.
When cases of monkeypox first appeared in the US, public health officials were concerned about the availability of testing, vaccines, and treatments. As CNN reported, though there was a new vaccine available, its effectiveness against monkeypox had never been tested on humans.
That treatment, known as TPOXX (Tecovirimat), was an antiviral drug approved by the FDA in 2018 to treat smallpox in adults and children, according to an FDA factsheet. The drug was difficult to obtain, and it took until August of 2022 for the federal government to declare monkeypox a public health emergency. That allowed it to deploy emergency funds towards fighting the outbreak.
The demographic found to be at the highest risk of monkeypox infection were men who have sex with other men. According to MedPage Today, “Daskalakis had both pandemic experience as former senior lead on equity in COVID-19 data and engagement for the New York City Department of Health and Mental Hygiene and an ‘in’ with the LGBTQ+ community from his work in HIV prevention and his transparency about being a gay man.”
When comparing monkeypox to HIV, Daskalakis said, “This one [monkeypox], you don’t have to change behaviors for generations; it’s for a few months. Once you build your force field of immunity with vaccines, people can make their own informed decisions about their risk.”
Opportunities for Clinical Laboratories
So, how should clinical laboratories respond if there’s another monkeypox flare up?
Daskalakis advocates for home testing. “People that are going to order home tests are going to be motivated to action in other ways. And so, thinking about HIV home testing, which was the grandparent of COVID-19 home testing, this really shows us how you reach people you’re not going to reach when you have lab-based, provider-only testing … When you look at the HIV home testing data from the CDC, 26% of the people that ordered a home test had never been tested before. That is way higher than what you would expect,” he told MedPage Today.
We are not out of the woods in regard to monkeypox, vigilance is still required. But with existing harm reduction measures in the most vulnerable community, at-home testing and advancements in vaccines could help us keep our numbers as low as possible.
Though clinical laboratory RT-PCR tests remain the ‘gold standard’ when diagnosing COVID-19, at-home antigen tests offer convenience and quick test results. But are they accurate?
Clinical laboratory and pathologists generally acknowledge that RT-Polymerase Chain Reaction (RT-PCR) tests remain the preferred method for detecting COVID-19 disease. However, according to Popular Science, rapid at-home antigen tests that accurately identify people carrying larger loads of the virus are becoming important tools in the fight against spread of the coronavirus.
DxTerity SARS-CoV-2: This saliva-based collection kit is available on Amazon and at Walmart for $110. Specimens must be sent via FedEx on the same day as collection. Results are available within 24 to 72 hours of the sample being received by the PCR testing lab.
Lucira Check It COVID-19 Test Kit: Billed as offering “PCR quality molecular accuracy,” this nasal swab test costs $55 and provides results in 30 minutes or less.
Abbott BinaxNOW Antigen Self-Test. Results within 15 minutes and available for $23.99 at major pharmacies. Each box includes two nasal swab tests, with the second test taken within 36 hours of the first.
Pixel COVID-19 At-Home Collection Kit: This nasal-swab-based mail-in collection kit is available online for $119. Results delivered within 24 to 48 hours of the sample being received by Labcorp.
Many of the newest at-home tests not only have users collect their own swab or saliva samples, but some also provide results in less than an hour, which can be sent to the user’s smartphone. Conversely, home-based collection kits that are returned to clinical laboratories for testing can take 48 hours or longer for shipping and processing.
The FDA’s emergency use authorization announcement (EUA) for Ellume’s $38.99 COVID-19 At Home Test (above) states the test “is a rapid, lateral flow antigen test, a type of test that runs a liquid sample along a surface with reactive molecules. The test detects fragments of proteins of the SARS-CoV-2 virus from a nasal swab sample from any individual two years of age or older.” Ellume’s self-collection test was the first such test to receive an FDA EUA for use without a physician’s order. (Photo copyright: Ellume).
Among the issues cited were the potential for inadequate samples and improper handling to cause inaccurate results, as well as uncertainty whether at-home antigen tests will pick up on COVID-19 variants.
At-home tests also are less likely to be covered by insurance, MedPage Today reported.
During the CAP virtual media briefing, pathologist Kalisha Hill, MD (above), Chief Medical Officer and Chair, Department of Pathology and Medical Director, Laboratory Services, at AMITA Health St. Mary’s Kankakee (Ill.), said, “The gold standard is still a laboratory-performed real-time PCR test and that is the most sensitive and most accurate that we do that is very specific for COVID-19.” Hill called at-home tests a “good screening tool,” but she noted, “You’re testing that moment, that day, and as soon as you leave your home or come in contact with someone else, you could potentially be COVID positive. It’s also important to recognize that when you’re collecting a sample yourself, you may not be able to obtain enough sample for an accurate result … It’s very important how it is collected and also the sensitivity and specificity of the test,” she added. (Photo copyright: AMITA Health/LinkedIn.)
How Do the Tests’ Accuracy Compare?
The Quest Direct and LabCorp Pixel tests—both of which are sent to company labs for PCR testing—scored highest on the two main statistical measures of performance sensitivity (positive percent agreement) and specificity (negative percent agreement). According to Popular Science, each of these tests’ sensitivities and specificities are close to 100%.
According to the websites of the other tests reviewed by Popular Science:
DxTerity test, which uses a saliva sample—97.2% sensitivity and a 92.5% specificity.
BinaxNOW test—84.6% sensitivity and 98.5% specificity.
Cue COVID-19 test—98.7% sensitivity and 97.6% specificity.
Lucira Check Its test—98% accuracy.
Ellume test—95% sensitivity and 97% specificity.
Rapid Antigen Tests Accurate and Easy to Use, says Popular Science
Popular Science found the tests generally easy to use and concluded they are a beneficial—if imperfect—tool in the fight against COVID-19.
“If you’re unvaccinated and symptomatic, they’re a great way to confirm a COVID-19 infection without risking a trip out of the house,” Popular Science stated in its article. “If you’re unvaccinated and have no symptoms, and just want to know whether you can safely attend a family dinner or soccer game, an at-home test remains an imperfect way of self-screening. Remember: If the test comes back negative, there’s still the chance the result is false, and you could accidentally expose others by being within six feet of them without a mask on.”
As the popularity of at-home COVID-19 tests increases, clinical laboratories that perform RT-PCR tests may want to keep a watchful eye on the demand for at-home rapid antigen testing, especially now that some tests are available without prescription.
CAP president maintains medical laboratory staff are ‘indispensable’ in pandemic fight and should be in ‘top tier’ for vaccination
As COVID-19 vaccinations continue to roll out, the College of American Pathologists (CAP) is lobbying for clinical pathologists and medical laboratory staff to be moved up the priority list for vaccinations, stating they are “indispensable” in the pandemic fight.
In a news release, CAP’s President Patrick Godbey, MD, FCAP argued for the early vaccination of laboratory workers, “It is essential that early access to the vaccine be provided to all pathologists and laboratory personnel,” he said. “Pathologists have led throughout this pandemic by bringing tests for the coronavirus online in communities across the country and we must ensure that patient access to testing continues. We must also serve as a resource to discuss the facts about the vaccine and answer questions patients, family members, and friends have about why they should get the vaccine when it is available to them.”
In a phone call following a virtual press conference, pathologists and CAP President Patrick Godbey, MD (above), told MedPage Today that even if medical laboratory staff are not directly in contact with patients, they should be considered “top tier” (designated as Phase 1a) for getting the vaccine. “I think they [clinical laboratory workers] should be considered in the same tier as nurses,” said Godbey, who also is Laboratory Director at Southeastern Pathology Associates and Southeast Georgia Health System in Brunswick, Ga. “They’re indispensable. Without them, there’d be no one to run the tests.” (Photo copyright: Southeast Georgia Health System.)
Who Does CDC Think Should Be First to Be Vaccinated?
According toThe New York Times (NYT), there are an estimated 21 million healthcare workers in the United States, making it basically “impossible,” the NYT wrote, for them all to get vaccinated in the first wave of COVID-19 vaccinations.
A December 11, 2020, CDC Morbidity and Mortality Weekly Report, titled, “ACIP Interim Recommendation for Allocating Initial Supplies of COVID-19 Vaccine—United States, 2020,” notes that “The [federal] Advisory Committee on Immunization Practices (ACIP) recommended, as interim guidance, that both 1) healthcare personnel and 2) residents of long-term care facilities be offered COVID-19 vaccine in the initial phase of the vaccination program.”
The ACIP report defines healthcare personnel as “paid and unpaid persons serving in healthcare settings who have the potential for direct or indirect exposure to patients or infectious materials.”
However, a CDC terminology guidance document listed at the bottom of the ACIP report states, “For this update, HCP [Healthcare Personnel] does not include dental healthcare personnel, autopsy personnel, and laboratory personnel, as recommendations to address occupational infection prevention and control (IPC) services for these personnel are posted elsewhere.”
In part, the letter stated, “We are convinced that ACIP did not intend to exclude any healthcare workers from its recommendation to offer vaccinations to healthcare personnel in the initial phase of the COVID-19 vaccination program (Phase 1a). However, we would hate for jurisdictions to overlook dental, autopsy, and laboratory personnel because of a minor footnote in [CDC] guidance that was developed for an entirely different purpose (i.e., infection control).
“We respectfully ask CDC to clarify,” the letter continues, “… that all healthcare workers—including dental, autopsy, and laboratory personnel—are among those who should be given priority access to vaccine during the initial phase of the COVID-19 vaccination program.”
“In the laboratory, they are encountering and handling thousands of samples that have active live virus in them,” said Karger, who called clinical laboratory staff and phlebotomists the “forgotten” frontline healthcare workers. “We’re getting 10,000 samples a day. That’s a lot of handling of infectious specimens, and we do want [staff] to be prioritized for vaccination.”
Karger continued to stress the vital role clinical laboratories play not only in COVID-19 testing but also in the functioning of the overall health system. She added that staff burnout is a concern since laboratory staff have been working “full throttle” since March.
“From an operational standpoint, we do need to keep our lab up and running,” she said. “We don’t want to have staff out such that we would have to decrease our testing capacity, which would have widespread impacts for our health system and state.”
Testing for Post-Vaccine Immunity
The CAP panelists also highlighted the need to prepare for the aftermath of widespread COVID-19 vaccinations—the need to test for post-vaccine immunity.
“It’s not routine practice to check antibody levels after getting a vaccine but given the heightened interest in COVID testing, we are anticipating there is going to be some increased in demand for post-vaccine antibody testing,” Karger said. “We’re at least preparing for that and preparing to educate our providers.”
Karger pointed out that clinical pathologists will play an important role in educating providers about the type of antibody tests necessary to test for COVID-19 immunity, because, she says, only the SARS-CoV-2 spike protein antibody test will check for an immune response.
With the pandemic expected to stretch far into 2021, clinical laboratories will continue to play a crucial role in the nation’s healthcare response to COVID-19. As essential workers in the fight against infectious disease, clinical pathologists, clinical chemists, and all medical laboratory staff should be prioritized as frontline healthcare workers.
