Researchers find a savings of more than one million dollars and prevention of hundreds, if not thousands, of adverse drug events could have been had with machine learning system
Support for artificial intelligence (AI) and machine learning (ML) in healthcare has been mixed among anatomic pathologists and clinical laboratory leaders. Nevertheless, there’s increasing evidence that diagnostic systems based on AI and ML can be as accurate or more accurate at detecting disease than systems without them.
Dark Daily has covered the development of artificial intelligence and machine learning systems and their ability to accurately detect disease in many e-briefings over the years. Now, a recent study conducted at Brigham and Women’s Hospital (BWH) and Massachusetts General Hospital (MGH) suggests machine learning can be more accurate than existing clinical decision support (CDS) systems at detecting prescription medication errors as well.
The study was partially retrospective in that the
researchers compiled past alerts generated by the CDS systems at BWH and MGH
between 2009-2011 and added them to alerts generated during the active part of
the study, which took place from January 1, 2012 to December 31, 2013, for a
total of five years’ worth of CDS alerts.
They then sent the same patient-encounter data that generated those CDS alerts to a machine learning platform called MedAware, an AI-enabled software system developed in Ra’anana, Israel.
MedAware was created for the “identification and prevention
of prescription errors and adverse drug effects,” notes the study, which goes
on to state, “This system identifies medication issues based on machine
learning using a set of algorithms with different complexity levels, ranging
from statistical analysis to deep learning with neural networks. Different
algorithms are used for different types of medication errors. The data elements
used by the algorithms include demographics, encounters, lab test results,
vital signs, medications, diagnosis, and procedures.”
The researchers then compared the alerts produced by
MedAware to the existing CDS alerts from that 5-year period. The results were
astonishing.
According to the study:
“68.2% of the alerts generated were unique to
the MedAware system and not generated by the institutions’ CDS alerting system.
“Clinical outlier alerts were the type least
likely to be generated by the institutions’ CDS—99.2% of these alerts were
unique to the MedAware system.
“The largest overlap was with dosage alerts,
with only 10.6% unique to the MedAware system.
“68% of the time-dependent alerts were unique to
the MedAware system.”
Perhaps even more important was the results of the cost
analysis, which found:
“The average cost of an adverse event
potentially prevented by an alert was $60.67 (range: $5.95–$115.40).
“The average adverse event cost per type of
alert varied from $14.58 (range: $2.99–$26.18) for dosage outliers to $19.14
(range: $1.86–$36.41) for clinical outliers and $66.47 (range: $6.47–$126.47)
for time-dependent alerts.”
The researchers concluded that, “Potential savings of $60.67 per alert was mainly derived from the prevention of ADEs [adverse drug events]. The prevention of ADEs could result in savings of $60.63 per alert, representing 99.93% of the total potential savings. Potential savings related to averted calls between pharmacists and clinicians could save an average of $0.047 per alert, representing 0.08% of the total potential savings.
“Extrapolating the results of the analysis to the 747,985
BWH and MGH patients who had at least one outpatient encounter during the
two-year study period from 2012 to 2013, the alerts that would have been fired
over five years of their clinical care by the machine learning medication
errors identification system could have resulted in potential savings of
$1,294,457.”
Savings of more than one million dollars plus the prevention
of potential patient harm or deaths caused by thousands of adverse drug events
is a strong argument for machine learning platforms in diagnostics and
prescription drug monitoring.
Researchers Say Current Clinical Decision Support Systems
are Limited
Machine learning is not the same as artificial intelligence. ML is a “discipline of AI” which aims for “enhancing accuracy,” while AI’s objective is “increasing probability of success,” explained Tech Differences.
Healthcare needs the help. Prescription medication errors cause patient harm or deaths that cost more than $20 billion annually, states a Joint Commission news release.
CDS alerting systems are widely used to improve patient
safety and quality of care. However, the BWH-MGH researchers say the current
CDS systems “have a variety of limitations.” According to the study:
“One limitation is that current CDS systems are rule-based and can thus identify only the medication errors that have been previously identified and programmed into their alerting logic.
“Further, most have high alerting rates with many false positives, resulting in alert fatigue.”
Commenting on the value of adding machine learning
medication alerts software to existing CDS hospital systems, the BWH-MGH
researchers wrote, “This kind of approach can complement traditional rule-based
decision support, because it is likely to find additional errors that would not
be identified by usual rule-based approaches.”
However, they concluded, “The true value of such alerts is
highly contingent on whether and how clinicians respond to such alerts and
their potential to prevent actual patient harm.”
Future research based on real-time data is needed before machine
learning systems will be ready for use in clinical settings, HealthITAnalytics
noted.
However, medical laboratory leaders and pathologists will
want to keep an eye on developments in machine learning and artificial
intelligence that help physicians reduce medication errors and adverse drug
events. Implementation of AI-ML systems in healthcare will certainly affect
clinical laboratory workflows.
Savvy medical laboratory managers conduct internal audits of processes involved in deficiency citations so they can uncover how deficiencies occur and help eliminate recurrences
One trend that places clinical laboratories at risk involves increased regulation of lab processes, along with more thorough accreditation inspections. Compared to past years, both developments mean more ways for lab assessors to find greater numbers of deficiencies.
However, leading laboratory accreditation and quality improvement experts say that many deficiencies could be avoided if lab leaders conducted their own internal audits and continuous quality improvement projects ahead of visits by accrediting authorities.
In an exclusive interview with Dark Daily, Randall Querry, Director of Government Relations at the American Association for Laboratory Accreditation (A2LA) said, “Clinical laboratories can do a better job of preparing for the external assessment by doing an internal audit. That is, watching personnel perform tests and noting if they aren’t following the same sequences that standard operating procedures address before the external assessors arrive.”
“This doesn’t have to be an ‘us against them’ exercise. We are
all in this together for continual improvement and to ensure we’re doing a
better job at the end of each day—that we have had a win,” said Querry
said.
How Should Clinical Laboratories Conduct Internal Audits?
So, what is the best method for clinical laboratory leaders to
conduct their own audits of operations and avoid citations of deficiencies?
Lucia Berte, President of Laboratories Made Better, suggested medical laboratories should “Pick a sequence and follow it through.” In the Dark Daily interview, she suggested labs should focus on:
The sequence of receiving samples in the
laboratory to make certain they are properly accessioned, processed, and
distributed;
Steps to setting up and running an analyzer; and
The process of ensuring tests’ critical values
are reported to ordering clinicians and how reports are made.
An internal audit may suggest areas where the clinical lab
is not on target to meet regulatory and accreditation criteria. Or, the lab may
discover what Querry calls “gray areas”—places where criteria are currently
being met, but a trend suggests there could be problems down the road.
“And in those cases, it’s always good to identify areas of
improvement for preventative action. They may not be a top priority—such as a
deficiency—but the areas are on the radar screen as something to address to prevent
it becoming a worsening problem,” Querry said.
Quality Improvement Processes to Address Deficiencies
Berte notes that citations in one area of the lab may
suggest the need for continuous improvement projects across all laboratory
departments or sections. For example, an accrediting body may cite chemistry
for a deficiency while hematology and other departments do okay. However, that
determination can be deceiving.
“There is always an underlying process. And the better
question for the clinical laboratory is ‘can we make an improvement project out
of this that can solve this problem not only for the area where it was cited,
but perhaps prevent this problem from occurring in other lab [departments]
prior to the next external accreditation assessments?’” Berte said.
Lack of Uniformity among a Clinical Laboratory’s
Departments
Berte says a common deficiency is “lack of a uniform
competency assessment program” for staff throughout the lab. Assessors expect
laboratory departments to have the same competency assessment in regard to
processes, records, and the way documents are created, she explained.
Lucia Berte (above), President of Laboratories Made Better, advises improving quality of documents as a project across the entire lab. “We still have a lot of silo-mentality in labs—where chemistry is different from hematology which is different from transfusion. Labs should have a uniform approach to the way their documents are written, and this is not necessarily the case,” she told Dark Daily. (Photo copyright: Whitehat Communications.)
Competency-related Citations
Berte also said competency-related citations may happen when
documents read by auditors are not in sync with what the officials see in the clinical
lab during inspections. “People not doing things in the order in which things
have to happen. That’s the disconnect.”
Querry, speaking from the perspective of an assessor, adds,
“We see a discrepancy and ask—do they have the appropriate work procedures with
them at the workstation? Is it accessible? Where is this discrepancy? We
identify it and then it’s up to the lab to address it—in training, and between
the written procedure and the process.”
Consistency, he says, is important especially in
organizations where staff rotate among lab areas and different shifts.
Quality System Essentials for Clinical Laboratories
The website for the Clinical and Laboratory Standards Institute (CLIA) states that implementing a quality management system in the lab involves use of “quality system essentials (QSEs).” QSEs are key to lab workflow, communication, and training. They include documents and records management, assessments, and continual improvement.
Querry emphasizes that trying to predict what the hot citations may be in 2020 is not as important as focusing on the technical competence of the lab and its resources.
“We are not out to play gotcha. We are going in there, looking
at all the systems, and doing a sampling of testing in various departments of
the lab. It’s up to the lab to show us it is technically competent to perform
those tests. And they have the equipment and records that the equipment has
been checked and calibrated and maintained. We have an examination process,” he
said.
Experts agree, clinical laboratories that prepare for
external assessments with internal audits and continuous improvement programs
may reduce deficiencies during inspections.
All labs face the challenge of coping with shrinking budgets and staffing shortages, which is why coaching, management observation, and continuous improvement initiatives are proving helpful to medical laboratories
It’s the biggest generational shift since baby boomers began working in clinical laboratories and anatomic pathology groups. Across the nation, labs are watching their most experienced and knowledgeable medical technologists and other lab scientists retire. The need to train their replacements while maintaining peak productivity and controlling costs is motivating lab leaders to adopt powerful new management methods.
Innovative lab administrators and pathologists recognize that automation and the ability to leverage the increasing amounts of data produced by today’s innovative diagnostic technologies and assays can only go so far in helping to compensate for declining revenues.
This is why one trend is quietly gaining momentum. There are many medical laboratories, pathology groups, and other diagnostics providers working to help their lab staffs create a culture of continuous, meaningful improvement. The stakes are great. Not only is this essential for financial sustainability, it can be the source of competitive advantage with physicians, patients, and payers in today’s increasingly competitive diagnostic market.
This is why many medical laboratories are turning to continuous improvement systems such as Lean to increase personnel skills, reduce waste, and make the most out of shrinking budgets and margins. Yet, without a solid foundation of staff trained in these methods and a framework of processes to encourage improvement, lean laboratory managers often struggle to see consistent, significant improvements.
Optimizing Staff Performance and Developing Improvement Processes with Coaching and Management Observation
Performance Coaching and Management Observation offer powerful tools for laboratory managers to reinforce improvement efforts. They encourage the success of personnel, leverage existing personnel to meet growing demands while maintaining service levels, and establish an effective foundation for Lean execution.
Benefits of effective coaching and management observation sessions for laboratories include:
Enhancement of laboratory manager performance and skills;
Improved retention of skilled labor and consistent improvement of personnel skills through individualized assessment and improved communication;
Establishing a method of creating and maintaining a culture of continuous improvement, while empowering staff across all levels of lab operations; and,
Creating a competitive edge on laboratories struggling to implement Lean processes and other optimizations in response to increased workloads, reduced staff, and tighter budgets.
Stephen Stone (left), Managing Director, Argent Global Services, and, Rita D’Angelo, PhD (right), President and CEO, D’Angelo Advantage, spoke at Lab Quality Confab in 2018 on the benefits of coaching and management observation sessions for clinical laboratories and implementing continuous improvement systems using Lean Production Methods. (Photo copyright: Dark Daily.)
“While many laboratories are familiar with management observation because of competency testing, few laboratories use coaching and management observation as part of their Lean efforts.” Stephen Stone, Managing Director at Argent Global Services told Dark Daily. “Coaching and management observation offers laboratories an effective means to not only increase throughput using existing staffing and encourage sustainable growth, but also increase retention of existing skilled personnel and reduce hiring costs.”
Speaking at Lab Quality Confab in Oct. 2018, Stone highlighted why these later benefits are increasingly important to laboratories. Citing data from LabTestingMatters and the American Society for Clinical Pathology (ASCP), he reported that while the job market for medical technologist and laboratory technician openings should increase by roughly 11,300 openings in 2018, fewer than 5,000 individuals are graduating each year from accredited training programs.
Lab Supervisor Retirements Projected to Exceed Staff Retirements
Of possibly greater concern, he goes on to point out, is that projected retirement rates for supervisors are higher than those of staff. This means laboratories which fail to focus on staff development and retention could face further issues in both leadership and staffing shortages should trends continue.
“Investing in and empowering your staff will improve productivity, improve quality, improve safety, and help laboratories to work toward goals as a cohesive team,” Rita D’Angelo, PhD, President and CEO at D’Angelo Advantage, LLC, told Dark Daily. “As a result, costs and waste drop significantly. Coaching and management observation alongside a culture of continuous improvement can help labs to overcome many of the staffing and budget obstacles faced today.”
The webinar will include essential coaching skills to help laboratory managers pass on the skills to serve as Lean champions to personnel and establish the foundation and structure for a lasting culture of improvement within the laboratory.
C-Level laboratory leadership, laboratory directors, managers and supervisors, and key members of continuous improvement teams also can use the interactive Q/A session following the webinar to gain answers to questions and concerns directly facing their laboratories’ efforts to develop continuous improvement processes or implement Lean methodologies.
(To register for this critical Jan. 16th webinar, click here. Or, copy and paste this URL into your browser: https://www.darkdaily.com/webinar/performance-coaching-and-management-observations-to-improve-productivity-and-efficiency-strengthening-the-skills-of-management-to-execute-a-lean-lab-transformation-2-2/.)
Experts say medical laboratories must overcome staffing challenges and develop a culture of continual inspection readiness
Clinical laboratories and anatomic pathology groups will be entering 2019 facing not only financial upheaval from Medicare Part B fee cuts and narrowing payer networks, but also an increasingly rigorous and ever-evolving regulatory environment.
Although the Clinical Laboratory Improvement Amendments (CLIA) requirements have not undergone major changes since debuting in 1992, the medical laboratory industry has become more complex and technology-laden, resulting in old rules being applied to emerging technologies.
“People can get lulled into this sense that we know what standards are and we’re meeting them, but what has happened is the rules haven’t really changed—the industry has,” says Nora L. Hess, MBA, MT(ASCP), Senior Consultant for Accumen, a lab quality improvement company in San Diego, CA. “Technology is now jumping ahead so fast that keeping up with it and understanding how the rules are going to be applied is challenging.”
Anne T. Daley, MS, CMQOE, CSSBB, CLC, MT, DLM (left), and Nora L. Hess, MBA, MT(ASCP), PMP (right), are Senior Consultants with Chi Solutions, an Accumen company. Together, they have extensive first-hand experience guiding clinical laboratories through the rigid and rigorous process of achieving inspection-ready status. (Photo copyrights: Chi Solutions, Inc.)
Why being ‘Inspection Ready’ Can Improve Performance and Increase Revenue
Striving to be “inspection ready” should be the goal of every clinical laboratory and pathology group. However, Daley notes labs typically operate with staffs that are stretched thin by retirements, illnesses, staff shortages, or ancillary demands on administrators’ time, caused by system-wide initiatives that range from electronic health record (EHR) rollouts to integration and consolidation of other labs.
“Most hospital laboratories are staffed at a level where they are working hard just to meet the daily patient care needs,” Daley says. “You add in all these additional projects and something has to give.”
In today’s challenging environment, Hess says laboratories that consistently perform well during the inspection process share several characteristics, including the following:
a culture of quality across the laboratory; and,
an emphasis on inspection readiness.
“They make inspection readiness a consistent focus across the entire laboratory,” Hess notes. “It becomes part of the calendar. It becomes part of what they talk about all the time. It is a part of their decision-making. It is hardwired into who they are and what they do … successful laboratories are making this part of their day-to-day activities, so it doesn’t get lost in the shuffle and it doesn’t get back-burnered.”
Hess and Daley note the types of deficiencies cited by accreditation agencies tend not to vary much from year to year. Testing personnel competency and proficiency testing are common themes, annually ranking high on top-10 lists of deficiencies found. However, laboratory directors increasingly are being singled out for issues related to qualifications and performance of responsibilities.
(To register for their Dec. 13th webinar or to order a DVD, click here. Or, copy and paste this URL into your browser: https://www.darkdaily.com/webinar/make-your-lab-assessment-ready-in-2019-know-the-most-common-deficiencies-in-accreditation-and-certification/).
Along with its assessment of the rate of errors in diagnosis, the IOM has a plan to improve, but will doctors accept the IOM’s advice, or continue business as usual?
Diagnostic errors in the American healthcare system is a problem that is now on the radar screen of policymakers at the Institute of Medicine (IOM). Pathologists and clinical laboratory professionals will welcome this development, because recommendations from the IOM carry weight with Congress.
Thus, should the IOM develop specific actions items intended to reduce medical errors, not only are these suggestions likely to involve more effective use of medical laboratory tests by physicians, but there is a strong probability that Congress might eventually write these recommendations into future healthcare legislation.
The Institute of Medicine is a division of the National Academies of Sciences, Engineering, and Medicine. The IOM recently convened a committee that released a list of recommendations to address the problem of diagnostic errors in medicine. Those recommendations, however, are running up against ingrained mindsets and overconfidence on the part of physicians who are reluctant to include decision-support technology in the diagnostic process. (more…)
