The AI tool predicted UC disease activity with 89% accuracy and inflammation at the biopsy site with 80% accuracy. Its ability to stratify risk of UC flare was on par with human pathologists.
“This tool in the near future will speed up, simplify, and standardize histological assessment of ulcerative colitis and provide the clinician with accurate prognostic information in real time,” co–lead author Marietta Iacucci, MD, PhD, from the University of Birmingham (England), and University College Cork (Ireland), said in an interview.
“The tool needs to be refined and further validated before it is ready for daily clinical practice. That work is ongoing now,” Dr. Iacucci said.
The researchers describe their advanced AI-based computer-aided detection tool in a study published online in Gastroenterology.
‘Strong’ performance
They used 535 digitized biopsies from 273 patients with UC (mean age, 48 years; 41% women) to develop and test the tool. They used a subset of 118 to train it to distinguish remission from activity, 42 to calibrate it, and 375 to test it. An additional 154 biopsies from 58 patients with UC were used to externally validate the tool.
The model also was tested to predict the corresponding endoscopic assessment and occurrence of flares at 12 months.
UC disease activity was defined by three different histologic indices: the Robarts Histopathology Index (RHI), the Nancy Histological Index (NHI), and the newly developed PICaSSO Histologic Remission Index (PHRI).
The AI tool had “strong diagnostic performance to detect disease activity” (PHRI > 0) with an overall area under the receiver operating characteristic curve of 0.87 and sensitivity and specificity of 89% and 85%, respectively.
The researchers note that, while the AI tool was trained for the PHRI, its sensitivity for RHI and NHI histologic remission/activity was also high (94% and 89%, respectively).
Despite the different mix of severity grades, the AI model “maintained a good diagnostic performance, proving its applicability outside the original development setting,” they reported.
The AI tool could also predict the presence of endoscopic inflammation in the biopsy area with about 80% accuracy.
“Though imperfect, this result is consistent with human-assessed correlation between endoscopy and histology,” the researchers noted.
The model predicted the corresponding endoscopic remission/activity with 79% and 82% accuracy for UCEIS and PICaSSO, respectively.
The hazard ratios for disease flare-up between the AI system and pathologists assessed by PHRI was similar (4.64 and 3.56, respectively), “demonstrating the ability of the computer to stratify the risk of flare comparably well to pathologists,” they added.
Both histology and outcome prediction were confirmed in the external validation cohort.
The AI system delivered results in an average of 9.8 seconds per slide.
Potential ‘game changer’
UC is a “complex condition to predict, and developing machine learning–derived systems to make this diagnostic job quicker and more accurate could be a game changer,” Dr. Iacucci said in a news release.
With refinement, the AI tool will have an impact on both clinical trials and daily practice, the researchers wrote. In clinical practice, histological reporting remains “largely descriptive and nonstandard, thus would greatly benefit from a quick and objective assessment. Similarly, clinical trials in UC could efficiently overcome costly central readings.”
Assessing and measuring improvement in endoscopy and histology are difficult parts of treating UC, said David Hudesman, MD, codirector of the Inflammatory Bowel Disease Center at New York University Langone Health.
“We do not know how much improvement is associated with improved long-term outcomes,” Dr. Hudesman said in an interview. “For example, does a patient need complete healing or is 50% better enough?” Dr. Hudesman was not involved with the current research.
“This study showed that AI can predict – with good accuracy – endoscopy and histology scores, as well as 1-year patient outcomes. If this is validated in larger studies, AI can help determine if we should adjust/change therapies or continue, which is very important,” he said.
This research was supported by the National Institute for Health Research Birmingham Biomedical Research Centre. Dr. Iacucci and Dr. Hudesman reported no relevant financial relationships.
A version of this article first appeared on Medscape.com.