associated with early use of emollients.
The document, published in November 2022, updates a February 2021 version, said Robert Boyle, MD, PhD, senior author of the Cochrane Review and a pediatric allergist at Imperial College London. “The differences were slight,” he told this news organization. “Mainly, we had a little more data about food allergy outcomes, which slightly strengthened the concern about a possible increase in food allergy with emollients; and we had some new genetic information, which allowed us to add some further interaction analyses and confirm that chromosome 11 intergenic variant rs2212434 doesn’t seem to impact the effect – or lack of effect – of emollient on eczema development.”
The updated Cochrane Review concludes that, “based on low‐ to moderate-certainty evidence, skin care interventions such as emollients during the first year of life in healthy infants are probably not effective for preventing eczema; may increase risk of food allergy; and probably increase risk of skin infection.”
The latest publication should strengthen clinicians’ confidence in not recommending emollient use for preventing AD in at-risk infants – however, that message is being diluted by a stream of contradictory conclusions from poor-quality systematic reviews, say Dr. Boyle and two coauthors. “It’s a systematic problem of people churning out endless systematic reviews without much rigor,” explained the lead author Maeve Kelleher, MD, from Children’s Health Ireland, Crumlin. There have been “misleading systematic reviews published, often in high-ranking journals,” agreed Dr. Boyle.
“I have been an advocate of systematic reviews for the last 20 years, but they have gone completely out of control,” added Hywel Williams, MD, PhD, another of the Cochrane Review coauthors, who is professor of dermato-epidemiology and codirector of the Centre of Evidence Based Dermatology, at Nottingham (England) University Hospitals NHS Trust. In an editorial, published last year, Dr. Williams even posed the question: “Are Dermatology Systematic Reviews Spinning Out of Control?” in which he blamed “the misrepresentation of study results” – which he calls “the sin of spin” – for degrading the quality of science in dermatology.
“The field has become a ‘sausage machine’ industry that undermines the value of systematic reviews in providing a summary of the best evidence to inform patient care,” he wrote. “Fewer systematic reviews are needed in dermatology,” but “better ones” are needed, he continued, calling for all systematic reviews to be registered prospectively, and reported according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.
Earlier this year, in a letter to the editor, Dr. Kelleher, Dr. Boyle, Dr. Williams, and several others outlined their concerns after a systemic review and meta-analysis was published, “which came to very different conclusions” than their Cochrane Review.
“It is quite common to see non-Cochrane reviews published in leading specialty journals, which interpret data in a more positive light than Cochrane reviews, which have assessed a similar dataset/topic,” Dr. Boyle said in the interview.
Such concerns also apply to the publication of another systematic review that was recently published. “Overall, early application of emollients is an effective strategy for preventing AD development in high-risk infants,” reported senior author Xiaojing Kang, MD, PhD, from People’s Hospital of Xinjiang Uygur Autonomous Region, Urumchi, China, and coauthors, who could not be reached for comment. In their discussion, the authors cite several criticisms of the Cochrane Review: that it included two meeting abstracts and two “ineligible” studies; did not do subgroup analysis of high-risk infants; did not look at different types of emollients; and did not examine the risk of food sensitization.
“A Cochrane Review can be quite a large and complex document to negotiate for those who are not very familiar with Cochrane’s methodology,” said Dr. Boyle. He dismissed the criticism, saying “we did do subgroup analysis of high risk infants, we did look at different types of emollient, and we did look at food sensitization and food allergy risk. We only included eligible studies. … Certainly we would include abstracts of trials, which are not reported in any other form, in order to capture as complete a picture.”
Ultimately, Dr. Boyle said, the discrepancy in conclusions between such systematic reviews and the Cochrane Review relates to quality of methodology. “Our Cochrane review was an individual participant data (IPD) meta-analysis, meaning that authors of the main trials in this area shared their original datasets with us,” he said in the interview. “This is the ‘gold standard’ in systematic reviews, and allowed us to check data/ query inconsistencies and to apply a single-analysis methodology across all studies. It also allowed us to undertake some analyses, which are just not possible in aggregate data analysis based on published work without IPD.”
The most recently published systematic review had no registered protocol, “so, there is no transparency about the methods used,” he noted. “It is free and simple to register a protocol – multiple websites such as PROSPERO, open science framework, and zenodo allow this,” he said “In the journal I edit, we use availability of a registered protocol as a marker of quality. We find that systematic reviews with no registered protocol are almost universally poor quality.”
Dr. Williams is a founding member and coordinating editor of the Cochrane Skin Group 1998 to 2017. Dr. Boyle was paid by Cochrane for senior editor work, until recently, and had no other relevant disclosures. Dr. Kelleher had no relevant disclosures.