mlsanityData to 14 Oct 2026

Model-based Lifelong Reinforcement Learning with Bayesian Exploration

arXiv 2210.11579 ↗First posted 20 Oct 2022cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

We propose a model-based lifelong reinforcement-learning approach that estimates a hierarchical Bayesian posterior distilling the common structure shared across different tasks. The learned posterior combined with a sample-based Bayesian exploration procedure increases the sample efficiency of learning across a family of related tasks. We first derive an analysis of the relationship between the sample complexity and the initialization quality of the posterior in the finite MDP setting. We next scale the approach to continuous-state domains by introducing a Variational Bayesian Lifelong Reinforcement Learning algorithm that can be combined with recent model-based deep RL methods, and that exhibits backward transfer. Experimental results on several challenging domains show that our algorithms achieve both better forward and backward transfer performance than state-of-the-art lifelong RL methods.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 29 of its 57 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

29 of 57 references.

References · 29

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2016Learning without Forgettingcited by 2,496
  5. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  7. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  9. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  10. 2014Convex Optimization: Algorithms and Complexitycited by 575
  11. 2015Weight Uncertainty in Neural Networkscited by 554
  12. 2016Deep Exploration via Bootstrapped DQNcited by 527
17 more
  1. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  2. 2016Learning to reinforcement learncited by 377
  3. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  4. 2018Probabilistic Model-Agnostic Meta-Learningcited by 256
  5. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
  6. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  7. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  8. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  9. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 178
  10. 2019Benchmarking Model-Based Reinforcement Learningcited by 162
  11. 2018Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RLcited by 81
  12. 2013Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizationscited by 70
  13. 2017Robust and Efficient Transfer Learning with Hidden-Parameter Markov Decision Processescited by 29
  14. 2020Lifelong Policy Gradient Learning of Factored Policies for Faster Training Without Forgettingcited by 29
  15. 2020Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learningcited by 29
  16. 2020MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Explorationcited by 25
  17. 2018Learning Invariances for Policy Generalizationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 2; 2025: 3; 2026: 2.

12 papers.

  1. 2023A Definition of Continual Reinforcement Learningcited by 101
  2. 2023Continual Learning: Applications and the Road Forwardcited by 59
  3. 2023TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Modelscited by 53
  4. 2025A Survey of Continual Reinforcement Learningcited by 18
  5. 2022The Effectiveness of World Models for Continual Reinforcement Learningcited by 17
  6. 2024Statistical Guarantees for Lifelong Reinforcement Learning using PAC-Bayes Theorycited by 7
  7. 2025Knowledge Retention for Continual Model-Based Reinforcement Learningcited by 7
  8. 2024Continual Offline Reinforcement Learning via Diffusion-based Dual Generative Replaycited by 6
  9. 2023Ensemble Value Functions for Efficient Exploration in Multi-Agent Reinforcement Learningcited by 5
  10. 2026Position: agentic AI orchestration should be Bayes-consistentcited by 3
  11. 2025Entropy-regularized Gradient Estimators for Approximate Bayesian Inferencecited by 0
  12. 2026Bayesian Online Model Selectioncited by 0