How would you explain a p-value to a nontechnical stakeholder?

Instruction: Give a plain-language explanation without claiming the p-value is the probability that the hypothesis is true.

Context: Explain p-values clearly and distinguish statistical evidence from effect size or business impact.

Updated

Example Answer

A p-value tells us how unusual a result at least this extreme would be if the null hypothesis and the test’s assumptions were true. A small p-value suggests the data are less compatible with that model. It does not tell us the probability that the hypothesis is true or that the result happened “by chance.”

Plain-language example

“If our model assumed the change had no effect, a p-value of 0.03 would mean results at least this extreme occur with probability 3% under that model. That is evidence to examine alongside the estimated effect and its confidence interval.”

I would then explain the size of the change, the uncertainty, and whether it matters to the decision. A small p-value alone does not prove causation, establish practical importance, or correct a flawed experiment.

Reference

American Statistical Association: p-value statement

Related Questions