Wall Street Journal reports IBM losing Watson-for-Oncology partners and clients, but scientists remain confident artificial intelligence will revolutionize diagnosis and treatment of disease
What happens when a healthcare revolution is overhyped? Results fall short of expectations. That’s the diagnosis from the Wall Street Journal (WSJ) and other media outlets five years after IBM marketed its Watson supercomputer as having the potential to “revolutionize” cancer diagnosis and treatment.
The idea that artificial intelligence (AI) could be used to diagnose cancer and identify appropriate therapies certainly carried with it implications for clinical laboratories and anatomic pathologists, which Dark Daily reported as far back as 2012. It also promised to spark rapid growth in precision medicine. For now, though, that momentum may be stalled.
“Watson can read all of the healthcare texts in the world in seconds,” John E. Kelly III, PhD, IBM Senior Vice President, Cognitive Solutions and IBM Research, told Wired in 2011. “And that’s our first priority, creating a ‘Dr. Watson,’ if you will.”
However, despite the marketing pitch, the WSJ investigation published in August claims IBM has fallen far short of that goal during the past seven years. The article states, “More than a dozen IBM partners and clients have halted or shrunk Watson’s oncology-related projects. Watson cancer applications have had limited impact on patients, according to dozens of interviews with medical centers, companies and doctors who have used it, as well as documents reviewed by the Wall Street Journal.”
Anatomic pathologists—who use tumor biopsies to diagnose cancer—have regularly wondered if IBM’s Watson would actually help physicians do a better job in the diagnosis, treatment, and monitoring of cancer patients. The findings of the Wall Street Journal show that Watson has yet to make much of a positive impact when used in support of cancer care.
The WSJ claims Watson often “didn’t add much value” or “wasn’t accurate.” This lackluster assessment is blamed on Watson’s inability to keep pace with fast-evolving treatment guidelines, as well as its inability to accurately evaluate reoccurring or rare cancers. Despite the more than $15 billion IBM has spent on Watson, the WSJ reports there is no published research showing Watson improving patient outcomes.
Lukas Wartman, MD, Assistant Professor, McDonnell Genome Institute at the Washington University School of Medicine in St. Louis, told the WSJ he rarely uses the Watson system, despite having complimentary access. IBM typically charges $200 to $1,000 per patient, plus consulting fees in some cases, for Watson-for-Oncology, the WSJ reported.
“The discomfort that I have—and that others have had with using it—has been the sense that you never know how much faith you can put in those results,” Wartman said.
Rudimentary Not Revolutionary Intelligence, STAT Notes
IBM’s Watson made headlines in 2011 when it won a head-to-head competition against two champions on the game show “Jeopardy.” Soon after, IBM announced it would make Watson available for medical applications, giving rise to the idea of “Dr. Watson.”
In a 2017 investigation, however, published on STAT, Watson is described as in its “toddler stage,” falling far short of IBM’s depiction of Watson as a “digital prodigy.”
“Perhaps the most stunning overreach is in [IBM’s] claim that Watson-for-Oncology, through artificial intelligence, can sift through reams of data to generate new insights and identify, as an IBM sales rep put it, ‘even new approaches’ to cancer care,” the STAT article notes. “STAT found that the system doesn’t create new knowledge and is artificially intelligent only in the most rudimentary sense of the term.”
STAT reported it had taken six years for data engineers and doctors to train Watson in just seven types of cancers and keep the system updated with the latest knowledge.
“It’s been a struggle to update, I’ll be honest,” Mark Kris, MD, oncologist at Memorial Sloan Kettering Cancer Center in New York and lead Watson trainer, told STAT. “Changing the system of cognitive computing doesn’t turn on a dime like that. You have to put in the literature, you have to put in the cases.” (Photo copyright: Physician Education Resource.)
Watson Recommended Unsafe and Incorrect Treatments, STAT Reported
In July 2018, STAT reported that internal documents from IBM revealed Watson had recommended “unsafe and incorrect” cancer treatments.
David Howard, PhD, Professor, Health Policy and Management, Rollins School of Public Health at Emory University, blames Watson’s failure in part to the dearth of high-quality published research available for the supercomputer to analyze.
“IBM spun a story about how Watson could improve cancer treatment that was superficially plausible—there are thousands of research papers published every year and no doctor can read them all,” Howard told HealthNewsReview.org. “However, the problem is not that there is too much information, but rather there is too little. Only a handful of published articles are high-quality, randomized trials. In many cases, oncologists have to choose between drugs that have never been directly compared in a randomized trial.”
Howard argues the news media needs to do a better job vetting stories touting healthcare breakthroughs.
“Reporters are often susceptible to PR hype about the potential of new technology—from Watson to ‘wearables’—to improve outcomes,” Howard said. “A lot of stories would turn out differently if they asked a simple question: ‘Where is the evidence?’”
Peter Greulich, a retired IBM manager who has written extensively on IBM’s corporate challenges, told STAT that IBM would need to invest more money and people in the Watson project to make it successful—an unlikely possibility in a time of shrinking revenues at the corporate giant.
“IBM ought to quit trying to cure cancer,” he said. “They turned the marketing engine loose without controlling how to build and construct a product.”
AI Could Still Revolutionize Precision Medicine
Despite the recent negative headlines about Watson, AI continues to offer the promise of one day changing how pathologists and physicians work together to diagnose and treat disease. Isaac Kohane, MD, PhD, Chairman of the Biomedical Informatics Program at Harvard Medical School, told Bloomberg that IBM may have oversold Watson, but he predicts AI one day will “revolutionize medicine.”
“It’s anybody’s guess who is going to be the first to the market leader in this space,” he said. “Artificial intelligence and big data are coming to doctors’ offices and hospitals. But it won’t necessarily look like the ads on TV.”
How AI and precision medicine plays out for clinical laboratories and anatomic pathologists is uncertain. Clearly, though, healthcare is on a path toward increased involvement of computerized decision-making applications in the diagnostic process. Regardless of early setbacks, that trend is unlikely to slow. Laboratory managers and pathology stakeholders would be wise to keep apprised of these developments.
—Andrea Downing Peck
Related Information:
IBM Pitched its Watson Supercomputer as a Revolution in Cancer Care. It’s Nowhere Close
IBM’s Watson Wins Jeopardy! Next Up: Fixing Health Care
IBM’s Watson Supercomputer Wins Practice Jeopardy Round
MD Anderson Cancer Center’s IBM Watson Project Fails, and So Did the Journalism Related to It
What Went Wrong with IBM’s Watson?
IBM’s Watson Failed Against Cancer but AI Still Has Promise
Pathologists Take Note: IBM’s Watson to Attack Cancer with Help of WellPoint and Cedars-Sinai
