How would you isolate the effects of two simultaneous app feature updates?
Instruction: Two independent feature flags, A and B, may change engagement. Eligible accounts can receive neither, either, or both. Design a study using seven-day active days per assigned account, including randomization, interaction, nonreturning users, logging checks and uncertainty. Explain the illustrative means 2.0, 2.3, 2.2 and 2.8 for neither, A, B and A+B.
Updated
Official answer available
Read the opening below, then unlock the full answer and practical guidance.
Randomize eligible accounts into four persistent groups: neither feature, A only, B only, and A+B. This 2×2 factorial design estimates each feature's effect and whether their combination differs from adding their separate effects. Define account eligibility before assignment and assess spillovers before choosing the unit.
Prespecify the primary outcome as active days per assigned account over seven...