How do you select the appropriate statistical test for your data?

Instruction: Describe the process of choosing a statistical test based on the characteristics of your dataset and the research question.

Context: This question assesses the candidate's ability to apply their knowledge of statistics to practical scenarios, selecting the correct test based on data type, distribution, and analysis objectives.

Updated

Official answer

Start with the estimand and study design: a mean difference, risk difference, distributional difference or another quantity. Identify the outcome type, assignment/sampling unit, pairing, repeated measurements and clustering before choosing a test.

Examples include a Welch test for independent group means under suitable conditions, a paired test for within-person differences, and a contingency-table method for categorical outcomes. Sparse counts may need an exact or calibrated method. A rank test generally targets a different question from a difference in means.

Check the assumptions relevant to that method, including dependence, variance, sample information and outlier influence. A normality p-value is not an automatic test-selection switch. Neither t-tests nor ANOVA are reserved for small samples, and chi-square or rank tests are not simply the “large sample” alternatives.

Prespecify tails, multiplicity and stopping where applicable. Report estimates and intervals alongside the test, conduct justified sensitivity checks, and state any limitation that the available design cannot resolve.

Related Questions