Stakeholder involvement, as we have defined it, was closely related to panel composition, which has been examined by others. For example, a link was found between panel composition and ratings of the appropriateness of procedures. Those who used a given procedure were more likely to rate it as appropriate than were those who did not use it.15,16 Murphy and coworkers found that “members of a specialty are more likely to advocate techniques that involve their special-ty.”17(p37) Savoie and colleagues, in their critical appraisal of guidelines for cholesterol testing, found that “the greater the involvement of clinical experts in the development process of the clinical practice guidelines, the less the recommendations reflected the research evidence.”9(p76) This is consistent with our finding that broader stakeholder involvement was associated with methodologic criteria being met more often.
In our study, guideline developers that did not use rigorous methods appeared more likely to promote aggressive intervention. This may be true for guidelines for conditions other than hypertension and hyperlipidemia. However, guideline developers also may introduce biases toward less aggressive recommendations, eg, purchasers of health services. The degree to which bias is likely and even the direction sometimes may be difficult to predict.
The quality among the guidelines we assessed was not associated with year of publication or the country where the guidelines were developed. The 6 guidelines fulfilling 5 or more of the quality criteria were not published more recently. The countries of origin for these 6 guidelines were Australia, Canada, France, the United Kingdom, and the United States.
There are strong logical reasons for users of guidelines to consider the methods used by guideline developers. Given the extent of disagreement among guidelines, it is necessary for users to understand the basis of those recommendations. This is only possible if guideline developers employ systematic methods and explicitly report the methods that were used. Our study provides empirical support of skepticism toward guidelines that have been developed without employing systematic methods.
· Acknowledgments ·
Wethank Signe Flottorp and Lena Nordheim for helping with the appraisal of non-English guidelines and Jonathan Lomas and Brian Hutchison who helped develop the idea for this study.