How should you compare health across second-hand marketplace groups of different sizes and ages?
Instruction: Explain which rates, cohorts and observation windows you would compare. Distinguish observed transactions from proxies when purchases happen off-platform.
Updated
Example answer
I would compare groups on both scale and the chance that an eligible participant succeeds. Total listings or messages show workload and scale, but large groups naturally generate more of them. Useful rates include the proportion of new listings receiving a genuine inquiry within seven days and, where verifiable, the proportion sold within a fixed window. State the eligible listing denominator and remove duplicates consistently.
Group age and listing age are different. Compare groups at the same weeks since launch, and compare listing cohorts only after each has had the full outcome window. An established group's all-time totals cannot fairly compete with a newly launched group's first week.
Segment by category and locality because purchase frequency, supply density and delivery constraints differ. Report group-level rates with sample sizes and uncertainty; do not let a handful of listings establish a ranking. Show an overall view too, using explicit weights rather than silently averaging differently sized groups.
If transactions occur off-platform, contact initiation and seller-reported sale status are proxies with missingness and reporting bias. Validate a sample rather than label them confirmed sales. Pair liquidity with fraud reports, unresolved disputes and user feedback; more messages may represent friction rather than success.
Follow-up
Would you simply divide every metric by membership? No. Use the population actually eligible for the action, such as mature listings for sell-through. Membership includes inactive users and can obscure the buyer and seller sides.