CRISIS IN SCIENCE? OR CRISIS IN STATISTICS! MIXED MESSAGES IN STATISTICS WITH IMPACT ON SCIENCE

Abstract

Gelman and Loken (2014) draw attention to a “statistical crisis in science” and describe
how risks with multiple p-values can be present even in the analysis of a single data set.
There is indeed a crisis, as p-values are everywhere, in science, engineering, medicine,
social science, health care, and the media; and conflicting results are misrepresenting the
importance of p-values, and indeed of many disciplines themselves. We argue that risks
of misinterpretation are widespread, but that the crisis is really in the discipline of statistics,
and starts with mixed messages about the meaning and usage of p-values. These
mixed messages then have downstream effects that seriously misinform scientific endeavours.
What are these mixed messages concerning p-values? And should statistics continue
with such messages that compromise the discipline? We discuss this and offer recommendations.