Many are submitted, but few are chosen.
Concerned about the quality of submitted research papers based on large surgical databases that are not accepted for publication, the editorial board of JAMA Surgery has taken the initiative by giving some pointers to would-be authors. The journal editors have published a 10-point checklist of dos and don’ts to address commonly seen problems with submitted manuscripts. In addition, JAMA Surgery collaborated with the Surgical Outcomes Club to commission a series of practical guides on the most widely used data sets in an effort to improve the quality of surgical database research. The Surgical Outcomes Club is a consortium of surgeons and scientists who work to advance health services and outcomes research in surgery that was launched in 2005 at the American College of Surgeons Clinical Congress.
Dr. Melina Kibbe
The authors noted that, although JAMA Surgery receives hundreds of submissions of retrospective studies of large surgical databases each year, most of these studies have flaws in the data analysis or use a hypothesis that the data sets cannot address. Hence, the editors do not send most of them out for peer review. “Of those that are sent out for peer review, many are recommended to be rejected by expert peer reviewers as they find major methodological flaws in the use of these otherwise powerful data sets,” the team wrote.
“Research using data sets can be very powerful as the research can address questions and hypotheses using large populations of people. However, the research can have many weaknesses. First, the research is only as good as the data collection for each data set. Second, the investigator needs to be familiar with the types of research questions and hypotheses that can be addressed with each data set. Third, the statistical methodology used to analyze the data is also imperative,” said Dr. Kibbe in an interview.
The checklist begins with a recommendation that the researchers develop a clear, concise hypothesis using established criteria – either FINER (for feasible, interesting, novel, ethical, relevant) or PICO (patient, population, or problem; intervention, prognostic factor, or exposure; comparison or intervention; outcome); the checklist then goes on to include compliance with institutional review board and data use agreements and to emphasize the importance of a clear take-home message that addresses policy or clinical implications.
The series comprises 11 two-page articles that aim to serve as practical guides for using each of the most widely used surgical data sets, starting with the National Inpatient Sample and ending with the Society of Thoracic Surgeons data set. Each article includes a bulleted list of the data set’s attributes, an explanation of its limitations, a history of the data set, an explanation of how the data is collected and what is unique about the set’s features, and statistical considerations researchers should take into account when analyzing the data.