mlsanityData to 14 Oct 2026

Accelerating Task Generalisation with Multi-Level Skill Hierarchies

arXiv 2411.02998 ↗First posted 5 Nov 2024cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Creating reinforcement learning agents that generalise effectively to new tasks is a key challenge in AI research. This paper introduces Fracture Cluster Options (FraCOs), a multi-level hierarchical reinforcement learning method that achieves state-of-the-art performance on difficult generalisation tasks. FraCOs identifies patterns in agent behaviour and forms options based on the expected future usefulness of those patterns, enabling rapid adaptation to new tasks. In tabular settings, FraCOs demonstrates effective transfer and improves performance as it grows in hierarchical depth. We evaluate FraCOs against state-of-the-art deep reinforcement learning algorithms in several complex procedurally generated environments. Our results show that FraCOs achieves higher in-distribution and out-of-distribution performance than competitors.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 23 of its 40 references that are in the map. First posted on arXiv 5 Nov 2024.

Builds on

23 of 40 references.

References · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  6. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  7. 2016The Option-Critic Architecturecited by 448
  8. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  9. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  10. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  11. 2020Phasic Policy Gradientcited by 123
  12. 2017Meta Learning Shared Hierarchiescited by 93
11 more
  1. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  2. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 67
  3. 2016Learning and Transfer of Modulated Locomotor Controllerscited by 59
  4. 2017Multi-Level Discovery of Deep Optionscited by 51
  5. 2019DAC: The Double Actor-Critic Architecture for Learning Optionscited by 43
  6. 2018Learning Abstract Optionscited by 31
  7. 2017Learnings Options End-to-End for Continuous Action Taskscited by 24
  8. 2019The Termination Criticcited by 24
  9. 2019Successor Options: An Option Discovery Framework for Reinforcement Learningcited by 23
  10. 2021Improving Zero-shot Generalization in Offline Reinforcement Learning using Generalized Similarity Functionscited by 17
  11. 2023Creating Multi-Level Skill Hierarchies in Reinforcement Learningcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Boosting Hierarchical Reinforcement Learning with Meta-Learning for Complex Task Adaptation
  2. 2022Powderworld: A Platform for Understanding Generalization via Rich Task Distributions
  3. 2024Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization
  4. 2025Scalable Option Learning in High-Throughput Environments
  5. 2026HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents
  6. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers
  7. 2024MENTOR: Guiding Hierarchical Reinforcement Learning with Human Feedback and Dynamic Distance Constraint
  8. 2024Hybrid Training for Enhanced Multi-task Generalization in Multi-agent Reinforcement Learning