Missing Data Multiple Imputation for Tabular Q-Learning in Online RL
On the map: Deep reinforcement learning methods → Deep RL algorithms and representations
Abstract (arXiv)
Missing data in online reinforcement learning (RL) poses challenges compared to missing data in standard tabular data or in offline policy learning. The need to impute and act at each time step means that imputation cannot be put off until enough data exist to produce stable imputation models. It also means future data collection and learning depend on previous imputations. This paper proposes fully online imputation ensembles. We find that maintaining multiple imputation pathways may help balance the need to capture uncertainty under missingness and the need for efficiency in online settings. We consider multiple approaches for incorporating these pathways into learning and action selection. Using a Grid World experiment with various types of missingness, we provide preliminary evidence that multiple imputation pathways may be a useful framework for constructing simple and efficient online missing data RL methods.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 31 references that are in the map. First posted on arXiv 12 Oct 2025.
Builds on
12 of 31 references.
References · 12
- 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 756
- 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 279
- 2014An Information-Theoretic Analysis of Thompson Samplingcited by 178
- 2014Learning to Optimize via Information-Directed Samplingcited by 168
- 2013Online Learning under Delayed Feedbackcited by 138
- 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
- 2022Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matterscited by 74
- 2020Discriminative Particle Filter Reinforcement Learning for Complex Partial Observationscited by 27
- 2020Blind Decision Making: Reinforcement Learning with Delayed Observationscited by 14
- 2024Imputation for prediction: beware of diminishing returnscited by 13
- 2019Robust Reinforcement Learning in POMDPs with Incomplete and Noisy Observationscited by 10
- 2024reBandit: Random Effects based Online RL algorithm for Reducing Cannabis Usecited by 6
Built on by
Citing papers per year: 2025: 1.
1 paper.