P-value

Lore & Background

In null-hypothesis significance testing, the p-value is defined as the probability, under the null hypothesis, of obtaining a real-valued test statistic at least as extreme as the one observed. For a one-sided right-tail test, this is Pr(T ≥ t | H₀); for a left-tail test, Pr(T ≤ t | H₀); and for a two-sided test, often 2 min{Pr(T ≥ t | H₀), Pr(T ≤ t | H₀)}.

Reader's Guide

The p-value is a cornerstone of null-hypothesis testing, but its interpretation is fraught with common errors. The p-value is a random variable dependent on the chosen test statistic; smaller p-values are generally taken as stronger evidence against the null hypothesis, but they do not indicate the magnitude or practical relevance of an effect. The significance level α is set by the researcher before examining the data, not derived from it.

Did You Know?

• The p-value is the probability of obtaining test results at least as extreme as the observed result, assuming the null hypothesis is true.
• Different p-values based on independent data sets can be combined using Fisher's combined probability test.