mlsanityData to 14 Oct 2026

Developing cooperative policies for multi-stage reinforcement learning tasks

arXiv 2205.05230 ↗First posted 11 May 2022cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Many hierarchical reinforcement learning algorithms utilise a series of independent skills as a basis to solve tasks at a higher level of reasoning. These algorithms don't consider the value of using skills that are cooperative instead of independent. This paper proposes the Cooperative Consecutive Policies (CCP) method of enabling consecutive agents to cooperatively solve long time horizon multi-stage tasks. This method is achieved by modifying the policy of each agent to maximise both the current and next agent's critic. Cooperatively maximising critics allows each agent to take actions that are beneficial for its task as well as subsequent tasks. Using this method in a multi-room maze domain and a peg in hole manipulation domain, the cooperative policies were able to outperform a set of naive policies, a single agent trained across the entire domain, as well as another sequential HRL algorithm.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 24 of its 30 references that are in the map. First posted on arXiv 11 May 2022.

Builds on

24 of 30 references.

References · 24

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  4. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  5. 2016The Option-Critic Architecturecited by 448
  6. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  7. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  8. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  9. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
  10. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  11. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  12. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 110
12 more
  1. 2019MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policiescited by 108
  2. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 98
  3. 2017Meta Learning Shared Hierarchiescited by 93
  4. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  5. 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 83
  6. 2019Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?cited by 68
  7. 2018Latent Space Policies for Hierarchical Reinforcement Learningcited by 67
  8. 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 46
  9. 2019Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewardscited by 40
  10. 2020Data-efficient Hindsight Off-policy Option Learningcited by 31
  11. 2019Hierarchical Soft Actor-Critic: Adversarial Exploration via Mutual Information Optimizationcited by 2
  12. 2019Hierarchical Reinforcement Learning for Concurrent Discovery of Compound and Composable Policiescited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 1.

4 papers.

  1. 2023Optimizing delegation between human and AI collaborative agentscited by 9
  2. 2024Optimizing Delegation in Collaborative Human-AI Hybrid Teamscited by 3
  3. 2024Optimizing Risk-averse Human-AI Hybrid Teamscited by 1
  4. 2025Causality-Based Reinforcement Learning Method for Multi-Stage Robotic Taskscited by 0