Extra

“Many of the conceptual difficulties that people have with NHST have their roots, I believe, in a failure to distinguish between absolute and conditional probabilities and, in particular, in failure to understand that the value of p produced by conventional tests of statistical significance is a conditional probability—the probability of getting the obtained statistical result on the assumption that the null is true” (Nickerson, 2000, p. 262) ^[1]

This article has a lot of misconceptions, especially regarding p-values

p-values are difficult to interpret because they are:

• Conditional probabilities: p = P (Data Results | H₀ = True)
  • And they are often incorrectly understood to be P (H₀ = True | Data Results)
• Influenced by:
  • sample size
  • effect size

Making decisions in the face of uncertainty. Uncertainty = sampling error.

Any decision based upon a hypothesis test can possibly be incorrect (Type I or Type II Error). Replication is the ultimate way to deal with sampling error. “No matter how intriguing a result from a single study, it must be replicated before it can be taken seriously.” ^[2] And statistical tests end up being unnecessary with enough replication. (All of this is Kline).

This has occurred, even though, as Steiger (1990) expressed: ‘An ounce of replication is worth a ton of inferential statistics’ (p. 176).