Clinical guidelines recommend use of exacerbation history in choosing therapies to predict the risk for chronic obstructive pulmonary disease exacerbations, but an analysis of data from three different clinical studies has found that exacerbation history alone is not the most accurate risk-prediction tool – and that it may even cause harm in some situations.
“Our results present a cautionary tale for the potential risk of harm to patients when naively applying risk stratification algorithms across different clinical settings,” lead author Joseph Khoa Ho, PharmD, a master’s candidate in pharmaceutical sciences at the University of British Columbia, Vancouver, told this news organization.
he said. “However, the prediction models required re-evaluation and setting-specific recalibration in order to yield higher clinical utility.”
The study, known as IMPACT, analyzed three trials that enrolled 4,107 patients at varying levels of moderate or severe exacerbation risks: the placebo arm of the Study to Understand Mortality and Morbidity in COPD (SUMMIT; N = 2,421); the Long-term Oxygen Treatment Trial (LOTT; N = 595); and the placebo arm of the Towards a Revolution in COPD Health trial (TORCH; N = 1,091). The exacerbation risks were low, medium, and high in the three respective trials.
The study, published online in the journal CHEST, compared the performance of three risk-stratification algorithms: exacerbation history; the model that Loes C.M. Bertens, PhD, and colleagues in the Netherlands developed in 2013; and the latest version of the Acute COPD Exacerbation Prediction Tool, known as ACCEPT.
Results of the analysis
The study used area under the curve (AUC), a method of evaluating effectiveness or efficiency, to compare performance of the prediction algorithms. ACCEPT outperformed exacerbation history and the Bertens algorithm in all the LOTT (medium risk) and TORCH (high risk) samples, both of which were statistically significant. In SUMMIT (low risk), Bertens and ACCEPT outperformed exacerbation history, which was statistically significant.
The AUC for exacerbation history alone in predicting future exacerbations in SUMMIT, LOTT, and TORCH was 0.59 (95% confidence interval, 0.57-0.61), 0.63 (95% CI, 0.59-0.67), and 0.65 (95% CI, 0.63-0.68), respectively. Bertens had a higher AUC, compared with exacerbation history alone in SUMMIT (increase of 0.10, P < .001) and TORCH (increase of 0.05, P < .001), but not in LOTT (increase of 0.01, P = .84).
ACCEPT had higher AUC, compared with exacerbation history alone in all study samples, by 0.08 (P < .001), 0.07 (P = .001) and 0.10 (P < .001), respectively. Compared with Bertens, ACCEPT had higher AUC by 0.06 (P = .001) in LOTT and 0.05 (P < .001) in TORCH, whereas the AUCs were not different in SUMMIT (change of –0.02, P = .16).
Study rationale
Senior author Mohsen Sadatsafavi, MD, PhD, associate professor of pharmaceutical sciences at the University of British Columbia, told this news organization that this study was inspired by a study in cardiology earlier in 2022 that found that the performance of the multitude of risk-prediction tools used to evaluate cardiovascular disease risk can vary widely if they’re not calibrated for new patient populations.
“The main finding was that exacerbation history alone can be harmful even if it is applied at different risk levels,” Dr. Sadatsafavi said of the IMPACT study. “No algorithm could be universally applicable, but exacerbation history has a very high chance of being worse than not doing any risk stratification at all and simply giving medication to all patients.”
Exacerbation history was considered harmful because it generated a lower net benefit than the either Bertens or ACCEPT, the IMPACT study found.
The benefit of the two risk-prediction tools is that they can be recalibrated, Dr. Sadatsafavi said. “You don’t have that luxury with exacerbation history, because it’s just a fixed positive or negative history,” he said. “We need to be quite cognizant of the difference in lung attacks in different populations and the fact that exacerbation history has very different performance in different groups and might be harmful when applied in certain populations. We suggest the use of the risk-stratification tools as a better proper statistical model.”
Expert comment
“As the authors point out, current guidelines for COPD management recommend preventive exacerbation therapy considering the patient’s exacerbation history,” Mary Jo S. Farmer, MD, PhD, assistant professor at the University of Massachusetts Chan Medical School-Baystate, Worcester, said via email. “However, this strategy has demonstrated harm in some situations.”
She noted that the multivariable prediction models were more accurate than exacerbation history alone for predicting 12-month risk of moderate/severe COPD exacerbations but that no algorithm was superior in clinical utility across all samples.
“The authors conclude that the highest accuracy of a risk prediction model can be achieved when the model is recalibrated based on the baseline exacerbation risk of the study population in question,” Dr. Farmer added.
The study received funding from the Canadian Institutes of Health Research. Dr. Ho, Dr. Sadatsafavi, and Dr. Farmer report no relevant financial relationships.
A version of this article first appeared on Medscape.com.