How do you interpret a 95% confidence interval?

Instruction: Explain what a 95% confidence interval means in the context of statistical analysis.

Context: This question tests the candidate's ability to understand and articulate the concept of confidence intervals and what they represent in terms of statistical certainty.

Updated

Example Answer

A 95% confidence interval comes from a procedure that would cover the true parameter in 95% of repeated samples, assuming its model and sampling assumptions hold. Once I have observed a particular interval, the frequentist statement is not that there is a 95% probability the fixed parameter lies inside it. That would require a different, Bayesian interpretation.

For example, suppose an A/B test estimates a conversion lift of one percentage point, with a 95% interval from minus half a point to plus two and a half points. The data are compatible with a small decline as well as an improvement under the analysis. I would compare that range with a practical decision threshold. The interval is not a range containing 95% of individual users’ outcomes, and crossing zero does not prove the variants are equivalent.

Why this works

It gives the repeated-sampling meaning and turns the interval into a decision-relevant range without claiming certainty.

Follow-up to prepare for

What would you need to conclude equivalence within a prespecified margin?

Related Questions