Advanced Causal Inference Techniques in Multi-armed Bandit Problems
Instruction: Discuss the application of causal inference methods in optimizing strategies for multi-armed bandit problems in real-time decision-making scenarios.
Context: This question explores how causal inference can be integrated into the decision-making process of multi-armed bandit algorithms. Explain how to implement causal techniques to improve the selection strategy, assess the impact of interventions on different arms, and how to handle the exploration-exploitation trade-off from a causal perspective.
Updated
Official answer available
Read the opening below, then unlock the full answer and practical guidance.
First distinguish learning a policy that maximizes reward from estimating a causal contrast or the value of another policy. In a contextual bandit, each record contains context, the selected action and its observed reward; rewards for unselected actions are missing.
Log the actual action probability conditional on the context and relevant prior history...