If further research confirms these findings, clinical laboratory identification of cancer cells could lead to new treatments for certain childhood cancers
Can cancer cells be changed into normal healthy cells? According to molecular biologists at the Cold Spring Harbor Laboratory (CSHL) in Long Island the answer is, apparently, yes. At least for certain types of cancer. And clinical laboratories and anatomic pathologists may play a key role in identifying these specific cancer cells and then guiding physicians in selecting the most appropriate therapies.
The cancer cells in question are called rhabdomyosarcoma (RMS) and are “particularly aggressive,” according to ScienceAlert. Generally, and most sadly, the cancer primarily affects children below the age of 18. It begins in skeletal muscle, mutates throughout the body, and is often deadly.
“Treatment usually involves chemotherapy, surgery, and radiation procedures. Now, new research by scientists at Cold Spring Harbor Laboratory demonstrates differentiation therapy as a new treatment option for RMS,” Genetic Engineering and Biotechnology News (GEN) reported.
For those young cancer patients, this new research could become a lifesaving therapy as further studies validate the approach, which has been in development for six years.
“Every successful medicine has its origin story,” said Christopher Vakoc, MD, PhD (above), a molecular biologist at Cold Spring Harbor Laboratory, who led the team that develop the method for converting cancer cells into healthy cells. “And research like this is the soil from which new drugs are born.” As these findings are confirmed, it may be that clinical laboratories and anatomic pathologists will be needed to identify the specific cancer cells in patients once treatment is developed. (Photo copyright: Cold Spring Harbor Laboratory.)
Differentiation Therapy
According to an article in the Chinese Journal of Cancer on the National Library of Medicine website, “Differentiation therapy is based on the concept that a neoplasm is a differentiation disorder [aka, differentiation syndrome] or a dedifferentiation disease. In response to the induction of differentiation, tumor cells can revert to normal or nearly normal cells, thereby altering their malignant phenotype and ultimately alleviating the tumor burden or curing the malignant disease without damaging normal cells.”
Vakoc and his team first pursued differentiation therapy to treat Ewing sarcoma, a pediatric cancer that forms in soft tissues or in bone. In January 2023, GEN reported that the researchers had discovered that “Ewing sarcoma could potentially be stopped by developing a drug that blocks the protein known as ETV6.”
“This protein is present in all cells. But when you perturb the protein, most normal cells don’t care,” Vakoc told GEN. “The process by which the sarcoma forms turns this ETV6 molecule—this relatively innocuous, harmless protein that isn’t doing very much—into something that’s now controlling a life-death decision of the tumor cell.”
The researchers discovered that when ETV6 was blocked in lab-grown Ewing sarcoma cells, the cells became normal, healthy cells. “The sarcoma cell reverts back into being a normal cell again,” they told GEN. “The shape of the cell changes. The behavior of the cells changes. A lot of the cells will arrest their growth. It’s really an explosive effect.”
The scientists then turned their attention on Rhabdomyosarcoma to see if they could elicit a similar response.
“In this study, we developed a high-throughput genetic screening method to identify genes that cause rhabdomyosarcoma cells to differentiate into normal muscle. We used this platform to discover the protein NF-Y as an important molecule that contributes to rhabdomyosarcoma biology. CRISPR-based genetic targeting of NF-Y converts rhabdomyosarcoma cells into differentiated muscle, and we reveal the mechanism by which this occurs,” they wrote in PNAS.
“Scientists have successfully induced rhabdomyosarcoma cells to transform into normal, healthy muscle cells. It’s a breakthrough that could see the development of new therapies for the cruel disease, and it could lead to similar breakthroughs for other types of human cancers,” ScienceAlert reported.
“The cells literally turn into muscle,” Vakoc told ScienceAlert. “The tumor loses all cancer attributes. They’re switching from a cell that just wants to make more of itself to cells devoted to contraction. Because all its energy and resources are now devoted to contraction, it can’t go back to this multiplying state,” he added.
Promising New Therapies for Multiple Cancers in Children
Differentiation therapy as a treatment option gained popularity when “scientists noticed that leukemia cells are not fully mature, similar to undifferentiated stem cells that haven’t yet fully developed into a specific cell type. Differentiation therapy forces those cells to continue their development and differentiate into specific mature cell types,” ScienceAlert noted.
Vakoc and his team had previously “effectively reversed the mutation of the cancer cells that emerge in Ewing sarcoma.” It was those promising results from differentiation therapy that inspired the team to push further and attempt success with rhabdomyosarcoma.
Their results are “a key step in the development of differentiation therapy for rhabdomyosarcoma and could accelerate the timeline for which such treatments are expected,” ScienceAlert commented.
Developing New Therapies for Deadly Cancers
Vakoc and his team are considering differentiation therapy’s potential effectiveness for other types of cancer as well. They note that “their technique, now demonstrated on two different types of sarcoma, could be applicable to other sarcomas and cancer types since it gives scientists the tools needed to find how to cause cancer cells to differentiate,” ScienceAlert reported.
“Since many forms of human sarcoma exhibit a defect in cell differentiation, the methodology described here might have broad relevance for the investigation of these tumors,” the researchers wrote in PNAS.
Clinical laboratories and anatomic pathologist play a critical role in identifying many types of cancers. And though any treatment that comes from the Cold Spring Harbor Laboratory research is years away, it illustrates how new insights into the basic dynamics of cancer cells is helping researchers develop effective therapies for attacking those cancers.
One goal of these new functions is to streamline physician workflows. However, these new EHRs may interface differently with clinical laboratory information systems
Artificial intelligence (AI) developers are making great contributions in clinical laboratory, pathology, radiology, and other areas of healthcare. Now, Electronic Health Record (EHR) developers are looking into ways to incorporate a new type of AI—called “Generative AI”—into their EHR products to assist physicians with time-consuming and repetitive administrative tasks and help them focus on patient-centered care.
Generative AI uses complex algorithms and statistical models to learn patterns from collected data. It then generates new content, including text, images, and audio/video information.
According to the federal Government Accountability Office (GAO), generative AI “has potential applications across a wide range of fields, including education, government, medicine, and law” and that “a research hospital is piloting a generative AI program to create responses to patient questions and reduce the administrative workload of healthcare providers.”
Reducing the workload on doctors and other medical personnel is a key goal of the EHR developers.
Generative AI uses deep learning neural networks modeled after the human brain comprised of layers of connected nodes that process data. It employs two neural networks: a generator [generative network] which creates new content, and a discriminator [discriminative network] which evaluates the quality of that content.
The collected information is entered into the network where each individual node processes the data and passes it on to the next layer. The last layer in the process produces the final output.
Many EHR companies are working toward adding generative AI into their platforms, including:
As our sister publication The Dark Report points out in its December 26 “Top 10 Biggest Lab Stories for 2023,” almost every product or service presented to a clinical laboratory or pathology group will soon include an AI-powered solution.
“We believe that generative AI has the potential of being a personal assistant for every doctor, and that’s what we’re working on,” Girish Navani (above), co-founder and CEO of eClinicalWorks, told EHRIntelligence. “It could save hours. You capture the essence of the entire conversation without touching a keyboard. It is transformational in how it works and how well it presents the information back to the provider.” Clinical laboratory information systems may also benefit from connecting with generative AI-based EHRs. (Photo copyright: eClinicalWorks.)
Generative AI Can Help with Physician Burnout
One of the beneficial features of generative AI is that it has the ability to “listen” to a doctor’s conversation with a patient while recording it and then produce clinical notes. The physician can then review, edit, and approve those notes to enter into the patient’s EHR record, thus streamlining administrative workflows.
“The clinician or support team essentially has to take all of the data points that they’ve got in their head and turn that into a narrative human response,” Phil Lindemann, Vice President of Data and Analytics at Epic, told EHRIntelligence. “Generative AI can draft a response that the clinician can then review, make changes as necessary, and then send to the patient.”
By streamlining and reducing workloads, EHRs that incorporate generative AI may help reduce physician burnout, which has been increasing since the COVID-19 pandemic.
“Language models have a huge potential in impacting almost every workflow,” Girish Navani, co-founder and CEO of eClinicalWorks, told EHRIntelligence. “Whether it’s reading information and summarizing it or creating the right type of contextual response, language models can help reduce cognitive load.”
Generative AI can also translate information into many different languages.
“Health systems spend a lot of time trying to make patient education and different things available in certain languages, but they’ll never have every language possible,” Lindemann said. “This technology can take human language, translate it at any reading level in any language, and have it understandable.”
MEDITECH is working on a generative AI project to simplify clinical documentation with an emphasis on hospital discharge summaries that can be very laborious and time-consuming for clinicians.
“Providers are asked to go in and review previous notes and results and try to bring that all together,” Helen Waters, Executive Vice President and COO of MEDITECH, told EHRIntelligence. “Generative AI can help auto-populate the discharge note by bringing in the discrete information that would be most relevant to substantiate that narrative and enable time savings for those clinicians.”
Many Applications for Generative AI in Healthcare
According to technology consulting and solutions firm XenonStack, generative AI has many potential applications in healthcare including:
The technology is currently in its early stages and does present challenges, such as lack of interpretability, the need for large datasets and more transparency, and ethical concerns, all of which will need to be addressed.
“We see it as a translation tool,” Lindemann told EHRIntelligence. “It’s not a panacea, but there’s going to be really valuable use cases, and the sooner the community can agree on that, the more useful the technology’s going to be.”
Since generative AI can be used to automate manual work processes, clinical laboratories and anatomic pathology groups should be alert to opportunities to interface their LISs with referring physicians’ EHRs. Such interfaces may enable the use of the generative AI functions to automate manual processes in both the doctors’ offices and the labs.
Norwegian researchers reviewed large clinical trials of six common cancer screenings, including clinical laboratory tests, but some experts question the findings
Cancer screenings are a critical tool for diagnosis and treatment. But how much do they actually extend the lives of patients? According to researchers at the University of Oslo in Norway, not by much. They recently conducted a review and meta-analysis of 18 long-term clinical trials, five of the six most commonly used types of cancer screening—including two clinical laboratory tests—and found that with few exceptions, the screenings did not significantly extend lifespans.
The 18 long-term clinical trials included in the study were randomized trials that collectively included a total of 2.1 million participants. Median follow-up periods of 10 to 15 years were used to gauge estimated lifetime gain and mortality.
“The findings of this meta-analysis suggest that current evidence does not substantiate the claim that common cancer screening tests save lives by extending lifetime, except possibly for colorectal cancer screening with sigmoidoscopy,” the researchers wrote in their published paper.
The researchers noted, however, that their analysis does not suggest all screenings should be abandoned. They also acknowledged that some lives are saved by screenings.
“Without screening, these patients may have died of cancer because it would have been detected at a later, incurable stage,” the scientists wrote, MedPage Today reported. “Thus, these patients experience a gain in lifetime.”
Still, some independent experts questioned the validity of the findings.
Gastroenterologist Michael Bretthauer, MD, PhD (above), a professor at the University of Oslo in Norway led the research into cancer screenings. In their JAMA Internal Medicine paper, he and his team wrote, “The findings of this meta-analysis suggest that colorectal cancer screening with sigmoidoscopy may extend life by approximately three months; lifetime gain for other screening tests appears to be unlikely or uncertain.” How their findings might affect clinical laboratory and anatomic pathology screening for cancer remains to be seen. (Photo copyright: University of Oslo.)
Pros and Cons of Cancer Screening
The clinical trials, according to MedPage Today and Oncology Nursing News covered the following tests:
Mammography screening for breast cancer (two trials).
As reported in these trials, “colorectal cancer screening with sigmoidoscopy prolonged lifetime by 110 days, while fecal testing and mammography screening did not prolong life,” the researchers wrote. “An extension of 37 days was noted for prostate cancer screening with prostate-specific antigen testing and 107 days with lung cancer screening using computed tomography, but estimates are uncertain.”
The American Cancer Society (ACS) recommends certain types of screening tests to detect cancers and pre-cancers before they can spread, thus improving the chances for survival.
The ACS advises screenings for breast cancer, colorectal cancer, and cervical cancer regardless of whether the individual is considered high risk. Lung cancer screenings are advised for people with a history of smoking. Men who are 45 to 50 or older should discuss the pros and cons of prostate cancer screening with their healthcare providers, the ACS states.
A CNN report about the University of Oslo study noted that the benefits and drawbacks of cancer screening have long been well known to doctors.
“Some positive screening results are false positives, which can lead to unnecessary anxiety as well as additional screening that can be expensive,” CNN reported. “Tests can also give a false negative and thus a false sense of security. Sometimes too, treatment can be unnecessary, resulting in a net harm rather than a net benefit, studies show.”
In their JAMA paper, the University of Oslo researchers wrote, “The critical question is whether the benefits for the few are sufficiently large to warrant the associated harms for many. It is entirely possible that multicancer detection blood tests do save lives and warrant the attendant costs and harms. But we will never know unless we ask,” CNN reported.
Hidden Impact on Cancer Mortality
ACS Chief Scientific Officer William Dahut, MD, told CNN that screenings may have an impact on cancer mortality in ways that might not be apparent from randomized trials. He noted that there’s been a decline in deaths from cervical cancer and prostate cancer since doctors began advising routine testing.
“Cancer screening was never really designed to increase longevity,” Dahut said. “Screenings are really designed to decrease premature deaths from cancer.” For example, “if a person’s life expectancy at birth was 80, a cancer screening may prevent their premature death at 65, but it wouldn’t necessarily mean they’d live to be 90 instead of the predicted 80,” CNN reported.
Dahut told CNN that fully assessing the impact of cancer screenings on life expectancy would require a clinical trial larger than those in the new study, and one that followed patients “for a very long time.”
“From its title, one would have expected this paper to be based on analysis of individual lifetime data. However, it is not,” he wrote in a compilation of expert commentary from the UK’s Science Media Center. “The paper’s conclusions are based on arithmetic manipulation of relative rates of all-cause mortality in some of the screening trials. It is therefore difficult to give credence to the claim that screening largely does not extend expected lifetime.”
He also questioned the inclusion of one particular trial in the University of Oslo study—the Canadian National Breast Screening Study—“as there is now public domain evidence of subversion of the randomization in this trial,” he added.
Another expert, Leigh Jackson, PhD, of the University of Exeter in the UK, described the University of Oslo study as “methodologically sound with some limitations which the authors clearly state.”
But he observed that “the focus on 2.1 million individuals is slightly misleading. The study considered many different screening tests and 2.1 million was indeed the total number of included patients, however, no calculation included that many people.”
Jackson also characterized the length of follow-up as a limitation. “This may have limited the amount of data included and also not considering longer follow-up may tend to underestimate the effects of screening,” he said.
This published study—along with the range of credible criticisms offered by other scientists—demonstrates how analysis of huge volumes of data is making it possible to tease out useful new insights. Clinical laboratory managers and pathologists can expect to see other examples of researchers assembling large quantities of data across different areas of medicine. This huge pools of data will be analyzed to determine the effectiveness of many medical procedures that have been performed for years with a belief that they are helpful.
Zoomers’ unique approach to work and personal health could affect clinical laboratory workplaces, how staff is managed, and how they personally use lab tests
Would it surprise you to learn that Generation Z is poised to make up 75% of the workforce in the United States by 2025? This fact has many implications for clinical laboratories, genetic testing companies, and pathology practices. That’s because Zoomers, as they are called, will be dominant in two ways. First, they will make up the majority of the lab workforce. Second, they will be the majority of consumers and patients accessing medical laboratory testing services.
Zoomers (born 1997-2012) approach work and their own healthcare differently than previous generations. This is partly due to Zoomers being “digital natives who have little or no memory of the world as it existed before smartphones,” according to Pew Research.
Now, a recently released report by economic research firm Glassdoor on 2024 workplace trends states that Zoomers are about to overtake Baby Boomers (born 1946-1964) in the full-time workforce, and that the shift will “represent a pivotal moment of cultural change that US companies cannot ignore.” This includes clinical laboratories and pathology groups that employ them.
According to Glassdoor, Gen Z workers “care deeply about community connections, about having their voices heard in the workplace, about transparent and responsive leadership, and about diversity and inclusion.”
Zoomers bring unique requirements and attitudes to the workplace, but they may also be the fresh infusion of talent a shrinking healthcare workforce needs. It’s no secret that clinical laboratories and pathology groups are facing a labor shortage. An aging workforce combined with burnout from the COVID-19 pandemic have left the entire healthcare industry scrambling for workers.
“A recent survey by Elsevier Health predicts that up to 75% of healthcare workers will leave the profession by 2025. And a 2020 study conducted by the Association of American Medical Colleges (AAMC) projected a shortfall of up to 139,000 physicians by 2033,” Medscape reported.
In “Clinical Laboratories Suffer During the Great Resignation,” Dark Daily noted other causes that are behind the abundance of open positions, such as early retirements, graduating individuals experiencing more specialized training programs, and a shift in the way the current working generation views employment.
Thus, the current healthcare workplace should not only expect unique challenges as Zoomers take over, but also changes that come with adapting to a smaller, younger workforce.
“[Gen Z] will pressure employers to establish a company’s purpose in a way that contributes to a better society and prioritize a company’s purpose along with profits,” Mark Beal (above), Assistant Professor of Professional Practice and Communication at Rutgers University, told Forbes. “Having succeeded at remote learning, they will influence an increased transition to hybrid and remote work as well as the four-day work week.” Clinical laboratories that understand Zoomers’ motivations will likely have more success integrating them into their workforce. (Photo copyright: Rutgers University.)
However, there could be unique challenges with a Zoomer workforce as well. According to Forbes, more than half of these new workers are willing to leave their jobs over “dissatisfaction with fulfillment (59%), professional development (57%), and providing value (53%).”
Although this may make some older workers scoff, each generation has entered the workforce with its own unique perspective based on personal values, and the workplace has shifted and changed to reflect the new workers. The same can be said of the clinical laboratory and pathology workforces.
The chart above shows the different generations as a proportion of the total population of the United States as of 2025. It dramatically illustrates why the largest number of working age individuals will be from Generation Z (aka, Zoomers). With their unique interests and traits, Zoomers will want their workplaces to be responsive in ways that are much different than the generations that preceded them. This will be equally true of how Gen Z accesses clinical laboratory testing services. (Graphic copyright: The Wall Street Journal.)
Gen Z Likes Automation
Another aspect to the increasing Zoomer workforce is Gen Z’s comfort with automated technology. Automation has always shifted how clinical laboratories work, and it can have great benefits for clinical pathology as well.
According to Today’s Clinical Lab, automation reduces error rates by more than 70% and reduces the time needed for each staff member per specimen by 10%.
However, the benefit does not come from automation replacing workers, rather that automated processes reduce repetitive work that takes time and attention away from workers. And, as noted, Gen Z workers tend to be extremely tech-savvy given the prevalence of technology in their lives.
Automation could fill gaps when it comes to labor shortages, not by replacing workers, but by helping adjust the workflow and avoiding worker burnout by automating tedious tasks. And Gen Z workers may be uniquely suited to engage with automated testing technologies.
Evolving Healthcare Workplaces
“The coming year will … represent a pivotal moment of cultural change that US companies cannot ignore as Gen Z workers—who care deeply about community connections, about having their voices heard in the workplace, about transparent and responsive leadership, and about diversity and inclusion—make up a rapidly growing share of the workforce,” the Glassdoor report stated, adding that 2024 “will test the robustness of workplace institutions,” The Hill reported.
Clinical laboratory managers and pathologists will be managing a multi-generational workforce, each with its own attributes and requirements. Thus, lab managers will need to reflect these difference in the management decisions they make and how they organize the laboratory workplace.
FDA says the move will make it easier to gain authorization for other clinical laboratory tests to utilize at-home collection kits
In another sign of how diagnostic testing is responding to changing consumer preferences, the US Food and Drug Administration (FDA) granted marketing authorization to LetsGetChecked for the company’s Simple 2 test for chlamydia and gonorrhea, which includes at-home collection of samples sent to the test developer’s clinical laboratories in the US and in Ireland.
This marks the first time the FDA has cleared a diagnostic test for either condition in which samples are collected at home. It’s also the first test with at-home sample collection to be authorized for any sexually transmitted infection (STI) other than HIV, the FDA said in a new release.
Samples are collected through a vaginal swab or urine sample. “Results are delivered online in approximately 2-5 days with follow-up virtual consultations and treatment available if needed,” the company press release states.
Previously authorized tests for the conditions required sample collection at the point of care. The company also offers telehealth and online pharmacy services.
“This authorization marks an important public health milestone, giving patients more information about their health from the privacy of their own home,” said Jeff Shuren, MD, JD (above), Director of the FDA’s Center for Devices and Radiological Health. “We are eager to continue supporting greater consumer access to diagnostic tests, which helps further our goal of bringing more healthcare into the home.” With this emphasis on at-home testing from the FDA, clinical laboratories in the US and Ireland will likely be processing more at-home collected samples. (Photo copyright: FDA.)
Simple 2 Process and Costs
Prior to collecting the sample, the user goes online to complete a questionnaire and activate the kit, the FDA news release notes.
LetsGetChecked, headquartered in New York City and Dublin, Ireland, says its US labs are CLIA– and CAP-certified. The company currently offers more than 30 at-home tests covering STIs, men’s health, women’s health, and COVID-19, at prices ranging from $89 to $249 per test.
The Simple 2 test costs $99, and is not covered by insurance, Verywell Health reported. Consumers can get discounts by subscribing to quarterly, semiannual, or annual tests.
New Regulatory Pathway
The FDA said it reviewed the test under its De Novo regulatory pathway, which is intended for “low- to moderate-risk devices of a new type,” according to the news release.
“Along with this De Novo authorization, the FDA is establishing special controls that define the requirements related to labeling and performance testing,” the agency stated. “When met, the special controls, in combination with general controls, provide a reasonable assurance of safety and effectiveness for tests of this type.”
This creates a new regulatory classification, the agency said, that will make it easier for similar devices to obtain marketing authorization.
Citing data from the federal Centers for Disease Control and Prevention (CDC), the FDA news release states that chlamydia and gonorrhea are the most common bacterial STIs in the US. The CDC estimates that there were 1.6 million cases of chlamydia and more than 700,000 cases of gonorrhea in 2021.
“Typically, both infections can be easily treated, but if left untreated, both infections can cause serious health complications for patients, including infertility,” the news release states. “Expanding the availability of STI testing can help patients get quicker results and access to the most appropriate treatment, ultimately helping to curb the rising rates of STIs.”
Experts Praise the FDA’s Authorization of the Lab Test
STI experts contacted by STAT said they welcomed the FDA’s move.
“There are many people who would like to be tested for STIs who may not know where to go or who have barriers to accessing medical care,” said Jodie Dionne, MD, Associate Professor of Medicine in the University of Alabama at Birmingham (UAB) Division of Infectious Diseases. “If we are going to do a better job of reaching more sexually active people for STIs … we need to be creative about how to get them tested and treated in a way that is highly effective and works for them.”
Family physician Alan Katz, MD, a professor at the University of Hawaii John A Burns School of Medicine, told STAT that the Hologic assay is also used by clinicians who treat people in remote locations to diagnose STIs and is regarded as being highly accurate.
“This option is exceptionally useful for individuals who live in rural areas or are geographically distanced from a clinic where STI testing can be done and there is no telehealth option available,” he told STAT.
With this latest move, the FDA is recognizing that it is time to give consumers more control over their healthcare. This is a signal to clinical laboratories that they should be developing their own strategies and offerings that serve consumers who want to order their own tests. Of course, many states still require a physician’s signature on lab test orders, but that is likely to change over time.
Research results call into question the safety and dependability of using artificial intelligence in medical diagnosis, a development that should be watched by clinical laboratory scientists
ChatGPT, an artificial intelligence (AI) chatbot that returns answers to written prompts, has been tested and found wanting by researchers at the University of Florida College of Medicine (UF Health) who looked into how well it could answer typical patient questions on urology. Not good enough according to the researchers who conducted the study.
AI is quickly becoming a powerful new tool in diagnosis and medical research. Some digital pathologists and radiologists use it for data analysis and to speed up diagnostic modality readings. It’s even been said that AI will improve how physicians treat disease. But with all new discoveries there comes controversy, and that’s certainly the case with AI in healthcare.
Many voices in opposition to AI’s use in clinical medicine claim the technology is too new and cannot be trusted with patients’ health. Now, UF Health’s study seems to have confirmed that belief—at least with ChatGPT.
The study revealed that answers ChatGPT provided “fell short of the standard expected of physicians,” according to a UF Health new release, which called ChatGPT’s answers “flawed.”
The questions posed were considered to be common medical questions that patients would ask during a visit to a urologist.
The researchers believes their study is the first of its kind to focus on AI and the urology specialty and which “highlights the risk of asking AI engines for medical information even as they grow in accuracy and conversational ability,” UF Health noted in the news release.
“I am not discouraging people from using chatbots,” said Russell S. Terry, MD (above), an assistant professor in the UF College of Medicine’s department of urology and the study’s senior author, in a UF Health news release. “But don’t treat what you see as the final answer. Chatbots are not a substitute for a doctor.” Pathologists and clinical laboratory managers will want to monitor how developers improve the performance of chatbots and other applications using artificial intelligence. (Photo copyright: University of Florida.)
UF Health ChatGPT Study Details
UF Health’s study featured 13 of the most queried topics from patients to their urologists during office visits. The researchers asked ChatGPT each question three times “since ChatGPT can formulate different answers to identical queries,” they noted in the news release.
The urological conditions the questions covered included:
The researchers then “evaluated the answers based on guidelines produced by the three leading professional groups for urologists in the United States, Canada, and Europe, including the American Urological Association (URA). Five UF Health urologists independently assessed the appropriateness of the chatbot’s answers using standardized methods,” UF Health noted.
Notable was that many of the results were inaccurate. According to UF Health, only 60% of responses were deemed appropriate from the 39 evaluated responses. Outside of those results, the researchers noted in their Urology paper, “[ChatGPT] misinterprets clinical care guidelines, dismisses important contextual information, conceals its sources, and provides inappropriate references.”
When asked, for the most part ChatGPT was not able to accurately provide the sources it referenced for its answers. Apparently, the chatbot was not programmed to provide such sources, the UF Health news release stated.
“It provided sources that were either completely made up or completely irrelevant,” Terry noted in the new release. “Transparency is important so patients can assess what they’re being told.”
Further, “Only 7 (54%) of 13 topics and 21 (54%) of 39 responses met the BD [Brief DISCERN] cut-off score of ≥16 to denote good-quality content,” the researchers wrote in their paper. BD is a validated healthcare information assessment questionnaire that “provides users with a valid and reliable way of assessing the quality of written information on treatment choices for a health problem,” according to the DISCERN website.
ChatGPT often “omitted key details or incorrectly processed their meaning, as it did by not recognizing the importance of pain from scar tissue in Peyronie’s disease. As a result … the AI provided an improper treatment recommendation,” the UF Health study paper noted.
Is Using ChatGPT for Medical Advice Dangerous to Patients?
Terry noted that the chatbot performed better in some areas over others, such as infertility, overactive bladder, and hypogonadism. However, frequently recurring UTIs in women was one topic of questions for which ChatGPT consistently gave incorrect results.
“One of the more dangerous characteristics of chatbots is that they can answer a patient’s inquiry with all the confidence of a veteran physician, even when completely wrong,” UF Health reported.
“In only one of the evaluated responses did the AI note it ‘cannot give medical advice’ … The chatbot recommended consulting with a doctor or medical adviser in only 62% of its responses,” UF Health noted.
For their part, ChatGPT’s developers “tell users the chatbot can provide bad information and warn users after logging in that ChatGPT ‘is not intended to give advice,’” UF Health added.
Future of Chatbots in Healthcare
In UF Health’s Urology paper, the researchers state, “Chatbot models hold great promise, but users should be cautious when interpreting healthcare-related advice from existing AI models. Additional training and modifications are needed before these AI models will be ready for reliable use by patients and providers.”
UF Health conducted its study in February 2023. Thus, the news release points out, results could be different now due to ChatGPT updates. Nevertheless, Terry urges users to get second opinions from their doctors.
“It’s always a good thing when patients take ownership of their healthcare and do research to get information on their own,” he said in the news release. “But just as when you use Google, don’t accept anything at face value without checking with your healthcare provider.”
That’s always good advice. Still, UF Health notes that “While this and other chatbots warn users that the programs are a work in progress, physicians believe some people will undoubtedly still rely on them.” Time will tell whether trusting AI for medical advice turns out well for those patients.
The study reported above is a useful warning to clinical laboratory managers and pathologists that current technologies used in ChatGPT, and similar AI-powered solutions, have not yet achieved the accuracy and reliability of trained medical diagnosticians when answering common questions about different health conditions asked by patients.
