Describe a situation where you would use a chi-square test.

Official Answer

Use a Pearson chi-square test of independence to assess whether two categorical variables are associated. For example, suppose independent users are randomly assigned to layouts A and B and each user contributes one pre-defined engagement category.

A hypothetical sample has counts (high, medium, low) of (30, 40, 30) for A and (45, 35, 20) for B. Each row totals 100; column totals are 75, 75 and 50. Under independence, each row's expected counts are (37.5, 37.5, 25).

Sum (observed - expected)^2 / expected over all six cells: the statistic is 5.3333, with (2 - 1)(3 - 1) = 2 degrees of freedom. The upper-tail p-value is approximately 0.0695. At a pre-specified 0.05 level, this sample does not reject independence; it does not prove that the layouts are equivalent.

Check independent observation units and sufficiently large expected cell counts for the approximation. Repeated sessions from the same user are not independent user observations. Sparse tables may require an appropriate exact or resampling method. An association test alone neither identifies which category differs nor establishes a practically valuable improvement; examine category proportions and uncertainty. Causal interpretation additionally depends on the study design.

Source

SciPy: chi-square contingency test.