# Dynamic Selection in Algorithmic Decision-making

arXiv: https://arxiv.org/abs/2108.12547 · HTML: https://mlsanity.com/paper/2108.12547/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 42 references that are in the map.
- First posted on arXiv 28 Aug 2021.

## Builds on (most cited first, 6 of 6)

- [Mostly Exploration-Free Algorithms for Contextual Bandits](https://mlsanity.com/paper/1704.09011/) (2017)
- [What Doubling Tricks Can and Can't Do for Multi-Armed Bandits](https://mlsanity.com/paper/1803.06971/) (2018)
- [Off-Policy Evaluation via Adaptive Weighting with Data from Contextual Bandits](https://mlsanity.com/paper/2106.02029/) (2021)
- [Deeply-Debiased Off-Policy Interval Estimation](https://mlsanity.com/paper/2105.04646/) (2021)
- [Efficient Counterfactual Learning from Bandit Feedback](https://mlsanity.com/paper/1809.03084/) (2018)
- [Asymptotic Theory for IV-Based Reinforcement Learning with Potential Endogeneity](https://mlsanity.com/paper/2103.04021/) (2021)

## Built on by (most cited first, 1 of 1)

- [Statistical Estimation of Confounded Linear MDPs: An Instrumental Variable Approach](https://mlsanity.com/paper/2209.05186/) (2022)

## Abstract (arXiv)

This paper identifies and addresses dynamic selection problems in online learning algorithms with endogenous data. In a contextual multi-armed bandit model, a novel bias (self-fulfilling bias) arises because the endogeneity of the data influences the choices of decisions, affecting the distribution of future data to be collected and analyzed. We propose an instrumental-variable-based algorithm to correct for the bias. It obtains true parameter values and attains low (logarithmic-like) regret levels. We also prove a central limit theorem for statistical inference. To establish the theoretical properties, we develop a general technique that untangles the interdependence between data and actions.