The series concludes with tips from the statistical editors of JAMA Surgery – Amy H. Kaji, MD, PhD, of Harbor–University of California Los Angeles Medical Center in Torrance; Alfred W. Rademaker, PhD, of Northwestern University, Chicago; and Terry Hyslop, PhD, of Duke University, Durham, N.C. – for performing statistical analysis of large data sets: “With bigger data, random signals may denote statistical significance, and precision may be incorrectly inferred because of narrow confidence intervals,” the statistical editors noted. “While many principles apply to all studies, the importance of these methodological issues is amplified in large, complex data sets.”
However, they noted that large data sets are prone to bias and measurement errors. “It is important to respect and acknowledge the limitations of the data,” the statistical team wrote. They also reprise the introductory editorial’s call for a clear hypothesis and take-home message. “The challenge with Big Data is that it requires a carefully thought-out research question and a transparent analytic strategy,” the statistical editors said.
Dr. Karl Y. Bilimoria
And there are pitfalls. Dr. Bilimoria noted, “We shouldn’t let the database define our research. We should instead be asking interesting questions and then seeking out a database that fits best to answer the question.” He said one particular problem that comes up often for reviewers is trying to discern how researchers arrived at the population of interest in a study. “A lot of inclusion and exclusions criteria are applied, and unless [the reviewer] can see the decisions that were made in the process, some fairly important biases can be introduced unintentionally. We as reviewers would like to be able to follow that exclusion pathway.”