mlsanityData to 14 Oct 2026

EVaDE : Event-Based Variational Thompson Sampling for Model-Based Reinforcement Learning

arXiv 2501.09611 ↗First posted 16 Jan 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Posterior Sampling for Reinforcement Learning (PSRL) is a well-known algorithm that augments model-based reinforcement learning (MBRL) algorithms with Thompson sampling. PSRL maintains posterior distributions of the environment transition dynamics and the reward function, which are intractable for tasks with high-dimensional state and action spaces. Recent works show that dropout, used in conjunction with neural networks, induces variational distributions that can approximate these posteriors. In this paper, we propose Event-based Variational Distributions for Exploration (EVaDE), which are variational distributions that are useful for MBRL, especially when the underlying domain is object-based. We leverage the general domain knowledge of object-based domains to design three types of event-based convolutional layers to direct exploration. These layers rely on Gaussian dropouts and are inserted between the layers of the deep neural network model to help facilitate variational Thompson sampling. We empirically show the effectiveness of EVaDE-equipped Simulated Policy Learning (EVaDE-SimPLe) on the 100K Atari game suite.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 36 of its 45 references that are in the map. First posted on arXiv 16 Jan 2025.

Builds on

36 of 45 references.

References · 36

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  3. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  4. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  5. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  6. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  7. 2017Deep Reinforcement Learning that Matterscited by 754
  8. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  9. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  10. 2016Deep Exploration via Bootstrapped DQNcited by 527
  11. 2019Model-Based Reinforcement Learning for Ataricited by 502
  12. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
24 more
  1. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  2. 2017Noisy Networks for Explorationcited by 266
  3. 2021Mastering Atari Games with Limited Datacited by 237
  4. 2017Stochastic Variational Video Predictioncited by 234
  5. 2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
  6. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 172
  7. 2017Parameter Space Noise for Explorationcited by 161
  8. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 124
  9. 2019When to use parametric models in reinforcement learning?cited by 118
  10. 2016Why is Posterior Sampling Better than Optimism for Reinforcement Learning?cited by 117
  11. 2014Generalization and Exploration via Randomized Value Functionscited by 112
  12. 2020Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planningcited by 86
  13. 2018Tensor2Tensor for Neural Machine Translationcited by 85
  14. 2018Efficient Exploration through Bayesian Deep Q-Networkscited by 67
  15. 2018Temporal Difference Variational Auto-Encodercited by 49
  16. 2015Bootstrapped Thompson Sampling and Deep Explorationcited by 40
  17. 2019Prioritized Sequence Experience Replaycited by 20
  18. 2016Path Integral Guided Policy Searchcited by 15
  19. 2017Variational inference for the multi-armed contextual banditcited by 15
  20. 2019Thompson Sampling via Local Uncertaintycited by 12
  21. 2019Scalable Thompson Sampling via Optimal Transportcited by 7
  22. 2020Importance of using appropriate baselines for evaluation of data-efficiency in deep reinforcement learning for Ataricited by 5
  23. 2021State-Aware Variational Thompson Sampling for Deep Q-Networkscited by 5
  24. 2018NADPEx: An on-policy temporally consistent exploration method for deep reinforcement learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Posterior Sampling for Deep Reinforcement Learning
  2. 2025Accelerating Residual Reinforcement Learning with Uncertainty Estimation
  3. 2024More Efficient Randomized Exploration for Reinforcement Learning via Approximate Sampling
  4. 2026Tail-Likelihood Reinforcement Learning
  5. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
  6. 2025Building surrogate models using trajectories of agents trained by Reinforcement Learning
  7. 2024Uncertainty Representations in State-Space Layers for Deep Reinforcement Learning under Partial Observability
  8. 2026Deep Reinforcement Learning with Buffered Quantile Objectives