Artificial intelligence performs BSS assessments with higher sensitivity and specificity than human diagnosticians
In a recent study conducted by scientists at Cedars-Sinai Medical Center in Los Angeles, researchers evaluated a smartphone application (app) that uses artificial intelligence (AI) to assess and characterize digital images of stool samples. The app, it turns out, matched the accuracy of participating gastroenterologists and exceeded the accuracy of study patients’ self-reports of stool specimens, according to a news release.
Though smartphone apps are technically not clinical laboratory tools, anatomic pathologists and medical laboratory scientists (MLSs) may be interested to learn how health information technology (HIT), machine learning, and smartphone apps are being used to assess different aspects of individuals’ health, independent of trained healthcare professionals.
The issue that the Cedars Sinai researchers were investigating is the accuracy of patient self-reporting. Because poop can be more complicated than meets the eye, when asked to describe their bowel movements patients often find it difficult to be specific. Thus, use of a smartphone app that enables patients to accurately assess their stools in cases where watching the function of their digestive tract is relevant to their diagnoses and treatment would be a boon to precision medicine treatments of gastroenterology diseases.
The scientists published their findings in the American Journal of Gastroenterology, titled, “A Smartphone Application Using Artificial Intelligence Is Superior to Subject Self-Reporting when Assessing Stool Form.”
“This app takes out the guesswork by using AI—not patient input—to process the images (of bowel movements) taken by the smartphone,” said gastroenterologist Mark Pimentel, MD (above), Executive Director of Cedars-Sinai’s Medically Associated Science and Technology (MAST) program and principal investigator of the study, in a news release. “The mobile app produced more accurate and complete descriptions of constipation, diarrhea, and normal stools than a patient could, and was comparable to specimen evaluations by well-trained gastroenterologists in the study.” (Photo copyright: Cedars-Sinai.)
Pros and Cons of Bristol Stool Scale
In their paper, the scientists discussed the Bristol Stool Scale (BSS), a traditional diagnostic tool for identifying stool forms into seven categories. The seven types of stool are:
- Type 1: Separate hard lumps, like nuts (difficult to pass).
- Type 2: Sausage-shaped, but lumpy.
- Type 3: Like a sausage, but with cracks on its surface.
- Type 4: Like a sausage or snake, smooth and soft (average stool).
- Type 5: Soft blobs with clear cut edges.
- Type 6: Fluffy pieces with ragged edges, a mushy stool (diarrhea).
- Type 7: Watery, no solid pieces, entirely liquid (diarrhea).
In an industry guidance report on irritable bowel syndrome (IBS)and associated drugs for treatment, the US Food and Drug Administration (FDA) said the BSS is “an appropriate instrument for capturing stool consistency in IBS.”
But even with the BSS, things can get murky for patients. Inaccurate self-reporting of stool forms by people with IBS and diarrhea can make proper diagnoses difficult.
“The problem is that whenever you have a patient reporting an outcome measure, it becomes subjective rather than objective. This can impact the placebo effect,” gastroenterologist Mark Pimentel, MD, Executive Director of Cedars-Sinai’s Medically Associated Science and Technology (MAST) program and principal investigator of the study, told Healio.
Thus, according to the researchers, AI algorithms can help with diagnosis by systematically doing the assessments for the patients, News Medical reported.
30,000 Stool Images Train New App
To conduct their study, the Cedars-Sinai researchers tested an AI smartphone app developed by Dieta Health. According to Health IT Analytics, employing AI trained on 30,000 annotated stool images, the app characterizes digital images of bowel movements using five parameters:
- BSS,
- Consistency,
- Edge fuzziness,
- Fragmentation, and
- Volume.
“The app used AI to train the software to detect the consistency of the stool in the toilet based on the five parameters of stool form, We then compared that with doctors who know what they are looking at,” Pimentel told Healio.
AI Assessments Comparable to Doctors, Better than Patients
According to Health IT Analytics, the researchers found that:
- AI assessed the stool comparable to gastroenterologists’ assessments on BSS, consistency, fragmentation, and edge fuzziness scores.
- AI and gastroenterologists had moderate-to-good agreement on volume.
- AI outperformed study participant self-reports based on the BSS with 95% accuracy, compared to patients’ 89% accuracy.
Additionally, the AI outperformed humans in specificity and sensitivity as well:
- Specificity (ability to correctly report a negative result) was 27% higher.
- Sensitivity (ability to correctly report a positive result) was 23% higher.
“A novel smartphone application can determine BSS and other visual stool characteristics with high accuracy compared with the two expert gastroenterologists. Moreover, trained AI was superior to subject self-reporting of BSS. AI assessments could provide more objective outcome measures for stool characterization in gastroenterology,” the Cedars-Sinai researchers wrote in their paper.
“In addition to improving a physician’s ability to assess their patients’ digestive health, this app could be advantageous for clinical trials by reducing the variability of stool outcome measures,” said gastroenterologist Ali Rezaie, MD, study co-author and Medical Director of Cedars-Sinai’s GI Motility Program in the news release.
The researchers plan to seek FDA review of the mobile app.
Opportunity for Clinical Laboratories
Anatomic pathologists and clinical laboratory leaders may want to reach out to referring gastroenterologists to find out how they can help to better serve gastro patients. As the Cedars-Sinai study suggests, AI smartphone apps can perform BSS assessments as good as or better than humans and may be useful tools in the pursuit of precision medicine treatments for patient suffering from painful gastrointestinal disorders.
—Donna Marie Pocius
Related Information:
Study: App More Accurate than Patient Evaluation of Stool Samples
Industry Guidance Report: Irritable Bowel Syndrome—Clinical Evaluation of Drugs
Artificial Intelligence-based Smartphone App for Characterizing Stool Form
AI Mobile App Improves on “Subjective” Patient-Reported Stool Assessment in IBS
Artificial Intelligence App Outperforms Patient-Reported Stool Assessments
