Anton wanted me to react to

Scientists rise up against statistical significance,a letter written by 3 people and signed by 800 others (which may look high on the street but it's really an

*insignificant*fraction of similar or better "scientists" in the world – surely millions). Two of the three authors have written a similar manifesto to a Nature subjournal in 2017. The signatories mostly do things like psychology, human behavior, epidemiology – mostly soft sciences. I see only 4 signatories with some "physics" on their lines and 2 of them are "biophysicists".

First, I found that text to be largely incoherent, indicating a not really penetrating thinking of the authors. There isn't any sequence of at least three sentences that I could fully subscribe to. If there is a seed of a possibly valid point, it's always conflated with some fuzzy negative attitudes to the very existence of "statistical significance" and I think that no competent scientist could agree with those assertions in their entirety.

Statistical significance may be misunderstood and used in incorrect sentences, including fallacies of frequently repeated types (I will discuss those later) and in this sense, it may be "abused", but the same is true for any other tool concept in science (and outside science). One may "abuse" the wave function, quantum gravity, a doublet, a microscope, or a cucumber, too, and this website is full of clarifications of the abuses of most of these notions. But just because people abuse these things doesn't mean that we may or we should throw the concepts (and gadgets) to the trash bin.

When it comes to the description of the "frequent abuse of statistical significance", I don't see a statistically significant positive correlation between their comments and my views – and the correlation is probably negative although I am not totally certain whether that correlation is statistically significant. ;-)

Clearly, I must start with this assertion that will also be the punch line of this blog post:

Sciences that have experimental portions and that are "hard sciences" at least to some extent simply cannot work without the concept.A proof why it's essential: All of science is about the search for the truth. One starts with guessing a hypothesis and testing it. Whether a hypothesis succeeds in describing data has to be determined. The process is known as the hypothesis testing. The result of that test has to be quantitative. It's called the \(p\)-value (or similar, more advanced quantities). The term "statistical significance" is nothing else than a human name for a \(p\)-value or a qualitative description of whether the \(p\)-value is low enough for the hypothesis to get a passing grade. The very existence of science is really connected with the existence of the concept of the statistical significance although a few centuries ago, the significance often used to be so high or low that the concept wasn't discussed explicitly at all.

This is a mostly theoretical physics blog but there are hundreds of comments about 3-sigma this and 4-sigma that. You couldn't really express these ideas "totally differently" (except for switching from sigmas to \(p\)-values or using synonyms). We simply need to quantify how reasonable it is to interpret an experiment as an experiment in which the Standard Model has apparently failed.

You may click at Statistical significance to see that the Wikipedia provides us with a perfectly sound and comprehensible definition – which doesn't indicate that there's anything

*controversial*about the concept itself. A statistically significant outcome is one that is unlikely to emerge according to the null hypothesis. That's why such a result makes it likely that there's something beyond the null hypothesis. This kind of the interpretation of the empirical data represents the building blocks of almost all the reasoning in quantitative enough empirical sciences!