ABSTRACT
Purpose To provide the best care, physicians must determine what published information is relevant, valid, and clinically useful. Patient-Oriented Evidence that Matters (POEMs) defines relevance as information that addresses clinical questions, measures clinical outcomes, and has the potential to change practice. The most useful clinical information is presented in terms of absolute risk reduction (ARR), number needed to treat (NNT), and number needed to harm (NNH). The purpose of this study was to estimate the percentage of drug treatment articles published in major medical journals that provide a calculated ARR, NNT, or NNH.
Methods We independently reviewed all drug treatment articles in 7 journals during a 6-month period for relevance, validity, and clinical usefulness. (Journals included Journal of the American Medical Association [JAMA], Archives of Internal Medicine [Arch Intern Med], British Medical Journal [BMJ], New England Journal of Medicine, Lancet, Obstetrics and Gynecology [Obstet Gynecol], and Pediatrics.) We assessed clinical usefulness by recording whether the articles reported ARR, NNT, or NNH.
Results Of the 995 articles we reviewed, only 2.4% met relevance criteria. Fewer than 1% of all drug therapy articles were POEMs with calculated ARR, NNT, or NNH. Arch Intern Med, JAMA, and BMJ published the most drug therapy POEMs: 33%, 20%, and 17%, respectively. JAMA, BMJ, and Obstet Gynecol were the only journals that published POEMs with clinically useful information.
Conclusions Most major journals that address primary care issues do not publish drug therapy POEMs; those that do rarely present information in a clinically useful manner. Editors should require authors to provide ARRs, NNTs, and NNHs to help clinicians provide the best medical care for their patients.
Medical professionals are inundated by new information, which some have described as an “information jungle.”1 Thousands of articles are published each year in hundreds of journals,2 adding to an ever-expanding knowledge base. One study suggests that the experienced primary care physician uses up to 2 million pieces of information each year to manage patients.3 To provide appropriate patient care, physicians must stay abreast of current medical knowledge.4 However, busy clinicians have little time to navigate the information jungle and sift through all of the data to determine what is relevant and clinically useful.
More than a decade has passed since Allen Shaughnessy and David Slawson developed the concept of Patient-Oriented Evidence that Matters (POEMs), “a summary of a valid piece of research that carries information that is important to patients and so to their doctors.”5 They developed a formula to classify research as a POEM: U=R×V/W, where U=usefulness of the information to doctors, R=relevance of the information to doctors, V=validity of the information, and W=work to access the information.1 The most useful information is both relevant and valid and takes little work to access. In 2002, the British Medical Journal (BMJ) proposed publishing one POEM a week based on the following criteria:
- It addresses a question that doctors encounter.
- It measures outcomes that doctors and their patients care about: symptoms, morbidity, quality of life, and mortality.
- It has the potential to change the way doctors practice.5
Even with the advent of POEMs, the true benefit of research and its application to clinical practice has yet to be determined. Physicians still have to decide which studies are valid, interpret the outcomes, and determine how they affect individual practice.
Research shows that clinicians, patients, and policy makers are more impressed by larger percentage differences than smaller ones.6 This fact is evident in the way trial results are presented in the news, by pharmaceutical representatives, and in journal articles. The relative risk reduction (RRR) is touted as suggesting either benefit or reduced harm, and the absolute numbers are largely under-reported. One study found that treatment effectiveness was perceived to be lower when the absolute risk reduction (ARR) rather than the RRR was reported.7 Perception of effectiveness decreased further when the number needed to treat (NNT) was presented.
The authors of the study concluded that ARR and NNT provide more concrete information than RRR about an intervention because they express efficacy “in a way which incorporates both the magnitude of the reduction of risk and the baseline risk without treatment.” They note that “because the exclusive reporting of relative risk may overstate the effectiveness of a treatment, actual event rates and absolute changes in risk should be reported.”7