How would you adapt recommendations when rewards arrive after market conditions change?
Instruction: Fictional shop: click feedback is immediate, purchases mature after seven days, and returns after 30 days. A promotion changes item availability halfway through the week. Specify reward attribution, eligible actions, policy comparison and handling of immature outcomes.
Updated
Worked answer
Record context, policy version, eligible candidate set, chosen item, exposure probability, event time and subsequent feedback. Attribute outcomes to the original exposure with a declared window; do not credit today's policy for purchases caused by last week's recommendations. Mark seven-day purchase and 30-day return outcomes immature until their windows close.
Clicks are an early proxy. A skipped item is ambiguous unless exposure and opportunity to act are known; neither a click nor a skip establishes long-term satisfaction. Define a value metric such as contribution after returns and keep early and mature series separate. During a promotion, filter unavailable items before selection and log the resulting candidate/action distribution.
A contextual bandit may suffice if the decision's relevant reward can be modeled without action-dependent later states. Use sequential RL only when there is a justified long-term state/action problem and adequate evidence. Off-policy comparison needs support and valid propensities; a newly available item may have none. Use constrained experimentation and mature-cohort analysis, with latency, complaint and return guardrails. Do not treat an apparent early reward increase as proof of net benefit.