mlsanityData to 7 Oct 2026

Continual Model-Based Reinforcement Learning with Hypernetworks

arXiv 2009.11997 ↗First posted 25 Sep 2020cs.LG cs.AI cs.ROcited by 41

Not on the map. Why?

Abstract (arXiv)

Effective planning in model-based reinforcement learning (MBRL) and model-predictive control (MPC) relies on the accuracy of the learned dynamics model. In many instances of MBRL and MPC, this model is assumed to be stationary and is periodically re-trained from scratch on state transition experience collected from the beginning of environment interactions. This implies that the time required to train the dynamics model - and the pause required between plan executions - grows linearly with the size of the collected experience. We argue that this is too slow for lifelong robot learning and propose HyperCRL, a method that continually learns the encountered dynamics in a sequence of tasks using task-conditional hypernetworks. Our method has three main attributes: first, it includes dynamics learning sessions that do not revisit training data from previous tasks, so it only needs to store the most recent fixed-size portion of the state transition experience; second, it uses fixed-capacity hypernetworks to represent non-stationary and task-aware dynamics; third, it outperforms existing continual learning alternatives that rely on fixed-capacity networks, and does competitively with baselines that remember an ever increasing coreset of past experience. We show that HyperCRL is effective in continual model-based reinforcement learning in robot locomotion and manipulation scenarios, such as tasks involving pushing and door opening. Our project website with videos is at this link https://rvl.cs.toronto.edu/blog/hypercrl

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 7; 2023: 8; 2024: 11; 2025: 10; 2026: 4.

41 papers.

Sort:

By area:

  1. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  2. 2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 58
  3. 2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 33
  4. 2022Disentangling Transfer in Continual Reinforcement Learningcited by 31
  5. 2024Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problemcited by 23
  6. 2023Is Model Ensemble Necessary? Model-based RL via a Single Model with Lipschitz Regularized Value Functioncited by 19
  7. 2022Hypernetworks for Zero-shot Transfer in Reinforcement Learningcited by 18
  8. 2025A Survey of Continual Reinforcement Learningcited by 16
  9. 2022The Effectiveness of World Models for Continual Reinforcement Learningcited by 15
  10. 2022Continual Learning from Demonstration of Robotics Skillscited by 11
  11. 2022Beyond Supervised Continual Learning: a Reviewcited by 11
  12. 2022Relay Hindsight Experience Replay: Self-Guided Continual Reinforcement Learning for Sequential Object Manipulation Tasks with Sparse Rewardscited by 10
29 more
  1. 2023Multi-agent Continual Coordination via Progressive Task Contextualizationcited by 10
  2. 2024HyperMARL: Adaptive Hypernetworks for Multi-Agent RLcited by 10
  3. 2022Towards Evaluating Adaptivity of Model-Based Reinforcement Learning Methodscited by 9
  4. 2024Continual Learning for Smart City: A Surveycited by 9
  5. 2025Continual Reinforcement Learning by Planning with Online World Modelscited by 8
  6. 2023Online Reinforcement Learning in Non-Stationary Context-Driven Environmentscited by 6
  7. 2023Scalable and Efficient Continual Learning from Demonstration via a Hypernetwork-generated Stable Dynamics Modelcited by 6
  8. 2025Knowledge Retention for Continual Model-Based Reinforcement Learningcited by 6
  9. 2024Learning Pareto Set for Multi-Objective Continuous Robot Controlcited by 5
  10. 2024Augmenting Replay in World Models for Continual Reinforcement Learningcited by 4
  11. 2024Beyond Imitation: A Life-long Policy Learning Framework for Path Tracking Control of Autonomous Drivingcited by 4
  12. 2025Hyper-GoalNet: Goal-Conditioned Manipulation Policy Learning with HyperNetworkscited by 4
  13. 2023Population-size-Aware Policy Optimization for Mean-Field Gamescited by 3
  14. 2025HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systemscited by 3
  15. 2023Effect of Optimizer, Initializer, and Architecture of Hypernetworks on Continual Learning from Demonstrationcited by 2
  16. 2024FedCore: Straggler-Free Federated Learning with Distributed Coresetscited by 2
  17. 2024Learning to Navigate in Mazes with Novel Layouts using Abstract Top-down Mapscited by 2
  18. 2024Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learningcited by 2
  19. 2025Capability-Aware Shared Hypernetworks for Flexible Heterogeneous Multi-Robot Coordinationcited by 2
  20. 2025When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks?cited by 2
  21. 2025A Continual Offline Reinforcement Learning Benchmark for Navigation Taskscited by 2
  22. 2023Domain Generalization Deep Graph Transformationcited by 1
  23. 2025HypeMARL: Multi-Agent Reinforcement Learning For High-Dimensional, Parametric, and Distributed Systemscited by 1
  24. 2026ARROW: Augmented Replay for RObust World modelscited by 1
  25. 2024Partial Models for Building Adaptive Model-Based Reinforcement Learning Agentscited by 0
  26. 2025An open dataset of neural networks for hypernetwork researchcited by 0
  27. 2026DISC: Decoupling Instruction from State-Conditioned Control via Policy Generationcited by 0
  28. 2026T3S: Improving Multi-Task Reinforcement Learning with Task-Specific Feature Selector and Schedulercited by 0
  29. 2026Amortized Low-Rank Adaptation for Model-Based Reinforcement Learningcited by 0