mlsanityData to 14 Oct 2026

Hierarchical Primitive Composition: Simultaneous Activation of Skills with Inconsistent Action Dimensions in Multiple Hierarchies

arXiv 2110.01833 ↗First posted 5 Oct 2021cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning has shown its effectiveness in various applications, providing a promising direction for solving tasks with high complexity. However, naively applying classical RL for learning a complex long-horizon task with a single control policy is inefficient. Thus, policy modularization tackles this problem by learning a set of modules that are mapped to primitives and properly orchestrating them. In this study, we further expand the discussion by incorporating simultaneous activation of the skills and structuring them into multiple hierarchies in a recursive fashion. Moreover, we sought to devise an algorithm that can properly orchestrate the skills with different action spaces via multiplicative Gaussian distributions, which highly increases the reusability. By exploiting the modularity, interpretability can also be achieved by observing the modules that are used in the new task if each of the skills is known. We demonstrate how the proposed scheme can be employed in practice by solving a pick and place task with a 6 DoF manipulator, and examine the effects of each property from ablation studies.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 23 of its 40 references that are in the map. First posted on arXiv 5 Oct 2021.

Builds on

23 of 40 references.

References · 23

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2016Progressive Neural Networkscited by 1,361
  3. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  4. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  5. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  6. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 168
  7. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 143
  8. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 127
  9. 2019MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policiescited by 108
  10. 2017Meta Learning Shared Hierarchiescited by 93
  11. 2017Control of a Quadrotor with Reinforcement Learningcited by 92
  12. 2018A Dissection of Overfitting and Generalization in Continuous Reinforcement Learningcited by 87
11 more
  1. 2016Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Criticcited by 81
  2. 2018Neural Modular Control for Embodied Question Answeringcited by 56
  3. 2018Modular meta-learningcited by 51
  4. 2019Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulationcited by 35
  5. 2018Hierarchical visuomotor control of humanoidscited by 30
  6. 2019Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitivescited by 30
  7. 2019Composing Task-Agnostic Policies with Deep Reinforcement Learningcited by 20
  8. 2017Learning to Compose Skillscited by 15
  9. 2021Towards Hierarchical Task Decomposition using Deep Reinforcement Learning for Pick and Place Subtaskscited by 14
  10. 2019Reinforcement Learning for Robotic Manipulation using Simulated Locomotion Demonstrationscited by 11
  11. 2020On Simple Reactive Neural Networks for Behaviour-Based Reinforcement Learningcited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 2.

3 papers.

  1. 2022Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learningcited by 56
  2. 2025XPG-RL: Reinforcement Learning with Explainable Priority Guidance for Efficiency-Boosted Mechanical Searchcited by 1
  3. 2025A Visual Reinforcement Learning-Based Separate Primitive Policy for Peg-in-Hole Taskscited by 0