VariBASed: Variational Bayes-Adaptive Sequential Monte-Carlo Planning for Deep Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Optimally trading-off exploration and exploitation is the holy grail of reinforcement learning as it promises maximal data-efficiency for solving any task. Bayes-optimal agents achieve this, but obtaining the belief-state and performing planning are both typically intractable. Although deep learning methods can greatly help in scaling this computation, existing methods are still costly to train. To accelerate this, this paper proposes a variational framework for learning and planning in Bayes-adaptive Markov decision processes that coalesces variational belief learning, sequential Monte-Carlo planning, and meta-reinforcement learning. In a single-GPU setup, our new method VariBASeD exhibits favorable scaling to larger planning budgets, improving sample- and runtime-efficiency over prior methods.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 57 references that are in the map. First posted on arXiv 21 Feb 2026.
Builds on
35 of 57 references.
References · 35
- 2017Attention Is All You Needcited by 59,932
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2022Simplified State Space Layers for Sequence Modelingcited by 776
- 2017A Distributional Perspective on Reinforcement Learningcited by 743
- 2022Efficiently Scaling Transformer Inferencecited by 529
- 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
- 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
23 more
- 2018Maximum a Posteriori Policy Optimisationcited by 318
- 2019A Theory of Regularized Markov Decision Processescited by 279
- 2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
- 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
- 2018Deep Variational Reinforcement Learning for POMDPscited by 122
- 2014Neural Variational Inference and Learning in Belief Networkscited by 116
- 2016Why is Posterior Sampling Better than Optimism for Reinforcement Learning?cited by 116
- 2023Structured State Space Models for In-Context Reinforcement Learningcited by 109
- 2017Variational Sequential Monte Carlocited by 93
- 2023Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programscited by 79
- 2020On the role of planning in model-based deep reinforcement learningcited by 57
- 2020Meta-trained agents implement Bayes-optimal agentscited by 37
- 2021Efficient Transformers in Reinforcement Learning using Actor-Learner Distillationcited by 35
- 2015Neural Adaptive Sequential Monte Carlocited by 34
- 2023Amortized Variational Inference: When and Why?cited by 28
- 2021Amortized Variational Inference for Simple Hierarchical Modelscited by 19
- 2022Critic Sequential Monte Carlocited by 16
- 2024Nesting Particle Filters for Experimental Design in Dynamical Systemscited by 12
- 2024KalMamba: Towards Efficient Probabilistic State Space Models for RL under Uncertaintycited by 9
- 2025Trust-Region Twisted Policy Improvementcited by 5
- 2024SPO: Sequential Monte Carlo Policy Optimisationcited by 4
- 2025Reflect-then-Plan: Offline Model-Based Planning through a Doubly Bayesian Lenscited by 4
- 2025Sequential Monte Carlo for Policy Optimization in Continuous POMDPscited by 2
Built on by
Not cited yet.