AI platform accurately predicts surgical complications say UF researchers
By Doug Bennett
SEE CHART: The MySurgeryRisk mobile app gives doctors detailed information on each complication (left) and risk predictions from the algorithm along with major factors leading to the prediction (right).
Gainesville, Florida - Complications after surgery can pose many challenges for both physicians and patients. Now, University of Florida researchers have confirmed their artificial intelligence system accurately helps doctors predict and manage these problems.
Researchers believe the system is unique in its ability to accurately predict postoperative complications by automatically acquiring patients’ medical data and delivering it to doctors’ mobile devices. The system, known as MySurgeryRisk, is at least as accurate as physicians in predicting surgical complications and sometimes more so, newly published findings show.
At the heart of MySurgeryRisk is an algorithm powered by machine learning, a type of artificial intelligence, or AI. UF Health researchers have spent years developing and testing the system. Using nearly seven years of data from more than 74,000 procedures involving approximately 58,000 adult patients, the system was consistently able to match surgeons’ accuracy in predicting surgical outcomes, the researchers found. An accurate AI system can benefit both doctors and patients by augmenting medical decision-making and reducing complications.
“This is really exciting because we validated the algorithms that we developed prospectively. It was important to show that we achieved good predictive performance,” said Tezcan Ozrazgat-Baslanti, Ph.D., a research assistant professor of medicine and co-developer of MySurgeryRisk.
More broadly, researchers say the system harnesses AI to process clinical data in real time, creating an “analytic pipeline” that pushes valuable results to surgeons’ mobile devices.
The latest findings are large scale, high-level evidence that the MySurgeryRisk system performs well in scenarios that closely resemble true clinical settings, said Azra Bihorac, M.D., senior associate dean for research affairs at the UF College of Medicine, a lead researcher on the MySurgeryRisk project and director of UF’s Intelligent Critical Care Center.
An algorithm improves on the current methods for predicting surgical complications, which Bihorac said are neither automated nor particularly sophisticated.
The system developed by Bihorac and her colleagues used massive amounts of anonymized data from patients’ electronic medical records, including medications, lab results and sociodemographic data, up to one year before surgery. It also collected crucial data such as vital signs during the surgical procedures. The algorithm then produced fast, accurate predictions of prolonged intensive care unit stays and mortality risk after procedures. It also calculated the risk of eight major postoperative complications, including sepsis, acute kidney injury, and cardiovascular and neurologic issues.
To validate the system’s prediction accuracy, the researchers tested various algorithm models with up to 135 variables, including clinical, pharmacy, lab and demographic data. They also compared MySurgeryRisk’s ability to predict surgical complications with outcomes projected by surgeons in 100 cases. The algorithm was significantly better than surgeons at predicting blood clots in veins. It was essentially equal to its human counterparts in forecasting the probability of neurological complications, delirium, sepsis and prolonged mechanical ventilation.
Having an AI system for predicting surgical outcomes offers myriad benefits for patients, physicians and the health care system as a whole. For patients, surgery and its risks are pervasive: The average American can expect to have seven operations during his or her life and about 1.5 million U.S. patients develop a complication from a surgery each year. Postoperative complications occur in as many as 32% of procedures and a major complication can cost as much as $11,000, the researchers noted. MySurgeryRisk may also prove particularly helpful for physicians who are less experienced at predicting postoperative complications, said Bihorac,
An AI system like MySurgeryRisk is meant to augment surgeons’ skill and experience as well as expedite their decision-making, Bihorac said. While planning surgery, the discovery of higher risks could prompt a doctor-patient conversation about whether surgery is truly appropriate.
It can also be a reliable bellwether: Knowing that a patient is at high risk of complications allows surgeons to be more strategic in the operating room, such as using individualized blood pressure management.
The system is also unique and valuable in the way it collects data and distributes real-time information to physicians. In an operating room, thousands of data points that are potentially relevant to surgical complications - such as information from ventilator monitors and anesthesia devices - now go uncollected. Bihorac and her colleagues are working on ways to gather that data to make MySurgeryRisk even more precise.
The new findings build on two other successful validations of MySurgeryRisk. In 2019, the researchers published a smaller study comparing surgical complication predictions from the algorithm with those from a group of 20 physicians. In the second study, they tested the algorithm’s accuracy at calculating the risk of eight major complications among more than 51,000 surgical patients at UF Health Shands Hospital between 2000 and 2010. The system predicted complications with accuracy ranging from 70% for wound complications to 80% for sepsis.
Developing and testing MySurgeryRisk has been a collaborative, interdisciplinary effort involving experts across the UF campus, Bihorac said. A group of 22 engineering and medical researchers initially helped to develop and test the algorithm. Co-authors of the latest paper include Yuanfang Ren, Ph.D., a postdoctoral associate with expertise in computer science and software engineering; Tyler J. Loftus, M.D., an assistant professor with the UF department of surgery; Parisa Rashidi, Ph.D., an associate professor with the J. Crayton Pruitt Family department of biomedical engineering and co-director of the UF Intelligent Critical Care Center; and Matthew Ruppert, a medical student and clinical research coordinator and data analyst with the UF College of Medicine. Researchers from the UF departments of anesthesiology, medicine and electrical and computer engineering contributed to the latest findings.
Next, the researchers want to expand validation testing of MySurgeryRisk at other sites around the state and make the system more widely available on mobile platforms, Bihorac said.