Rethinking Data Analysis in Basic Biomedical Research

Abstract

The inability to reproduce key scientific results is a growing concern among scientists, funding agencies, academic journals and the public. Studies examining research practices in basic biomedical research have highlighted problems with study design, inadequate reporting of methods and results, and errors in statistical analyses and data visualization, even among papers published in high impact journals. Misuse of statistical methods includes using incorrect or suboptimal tests, summarizing data that were analyzed by nonparametric techniques as mean and standard deviation or standard error, reporting p-values that are inconsistent with the test statistic, p-hacking, and analyzing nonindependent data as though they are independent. Additional problems arise from inadequate reporting of statistical methods. This may include failing to provide a power calculation, not reporting which statistical test was used, or not providing adequate detail about the test, not addressing whether the assumptions of the statistical tests were examined, or not specifying how replicates were treated in the analysis. Finally, other researchers have focused on the need to reconsider current statistical practices. The reliance on null hypothesis testing and p-values has been heavily questioned, and researchers have proposed a variety of alternate approaches. A necessity for a fundamental change in the data visualization techniques was also suggests based on compelling evidence from studies examining figures that are used to present key scientific results. Along with addressing these problems, the scientific community is increasingly recognizing the need for better statistical education for basic scientists. These observations have sparked discussion about the respective roles and responsibilities of authors, peer reviewers and journals in improving the quality of scientific literature in basic biomedical research.

Slides

About the lecturer

Natasa Milic is an Associate Professor of Medical Statistics and Informatics at Medical Faculty University of Belgrade and Research Collaborator of Mayo Clinic, Rochester, USA. Her interest in biomedical research extends to more than a decade, and includes a broad spectrum of research areas such as: basic biomedical science research; design and analysis of clinical trials; community-based research; systematic reviews and meta-analysis; decision analysis; health technology assessment; and most recently meta-research. With her collaborative group she has showed that misuse of statistical methods is common in biomedical science research, even among papers published in high impact journals. As these problems stem from a limited understanding of statistics, her special interest is in identifying opportunities for improving biostatistics education by developing tools and strategies to promote education and dissemination of statistical knowledge in broader scientific community.