Design a system for reviewing and acting on experiment results at scale.

Instruction: Design the operating system. Cover inputs, review cadence, decision rules, owners, tradeoffs, and success metrics.

Context: Tests whether the candidate can design a repeatable operating model with decision rules, tradeoffs, and measurable outcomes.

Updated

Official answer available

Read the opening below, then unlock the full answer and practical guidance.

At scale, I would require a registry for every proposed, running, stopped and completed experiment, including assignment, metric/analysis plan, owner, exposure, status and results. A platform owner maintains registry/data integrity, analysts own validity review, experiment teams own implementation and followups, and accountable product owners approve launch decisions.

Automated checks would flag...

Upgrade to view official answer

Related Questions