mlsanityData to 14 Oct 2026

Hierarchical Orchestra of Policies

arXiv 2411.03008 ↗First posted 5 Nov 2024cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Continual reinforcement learning poses a major challenge due to the tendency of agents to experience catastrophic forgetting when learning sequential tasks. In this paper, we introduce a modularity-based approach, called Hierarchical Orchestra of Policies (HOP), designed to mitigate catastrophic forgetting in lifelong reinforcement learning. HOP dynamically forms a hierarchy of policies based on a similarity metric between the current observations and previously encountered observations in successful tasks. Unlike other state-of-the-art methods, HOP does not require task labelling, allowing for robust adaptation in environments where boundaries between tasks are ambiguous. Our experiments, conducted across multiple tasks in a procedurally generated suite of environments, demonstrate that HOP significantly outperforms baseline methods in retaining knowledge across tasks and performs comparably to state-of-the-art transfer methods that require task labelling. Moreover, HOP achieves this without compromising performance when tasks remain constant, highlighting its versatility.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 19 references that are in the map. First posted on arXiv 5 Nov 2024.

Builds on

12 of 19 references.

References · 12

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Intriguing properties of neural networkscited by 5,155
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2016Progressive Neural Networkscited by 1,361
  6. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  7. 2017Continual Learning with Deep Generative Replaycited by 1,158
  8. 2018Experience Replay for Continual Learningcited by 1,095
  9. 2019Three scenarios for continual learningcited by 614
  10. 2018Progress & Compress: A scalable framework for continual learningcited by 520
  11. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  12. 2023Policy Optimization via Adv2: Adversarial Learning on Advantage Functionscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning
  2. 2025Self-Composing Policies for Scalable Continual Reinforcement Learning
  3. 2026Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning
  4. 2024Continual Task Learning through Adaptive Policy Self-Composition
  5. 2026Brain-Inspired Hierarchical Modularity for General Continual Learning
  6. 2025Learning with Expert Abstractions for Efficient Multi-Task Continuous Control
  7. 2025A Continual Offline Reinforcement Learning Benchmark for Navigation Tasks
  8. 2025A Survey of Continual Reinforcement Learning