mlsanityData to 14 Oct 2026

Efficient Multi-Task Reinforcement Learning with Cross-Task Policy Guidance

arXiv 2507.06615 ↗First posted 9 Jul 2025cs.LG cs.AIcited by 3

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Multi-task reinforcement learning endeavors to efficiently leverage shared information across various tasks, facilitating the simultaneous learning of multiple tasks. Existing approaches primarily focus on parameter sharing with carefully designed network structures or tailored optimization procedures. However, they overlook a direct and complementary way to exploit cross-task similarities: the control policies of tasks already proficient in some skills can provide explicit guidance for unmastered tasks to accelerate skills acquisition. To this end, we present a novel framework called Cross-Task Policy Guidance (CTPG), which trains a guide policy for each task to select the behavior policy interacting with the environment from all tasks' control policies, generating better training trajectories. In addition, we propose two gating mechanisms to improve the learning efficiency of CTPG: one gate filters out control policies that are not beneficial for guidance, while the other gate blocks tasks that do not necessitate guidance. CTPG is a general framework adaptable to existing parameter sharing approaches. Empirical evaluations demonstrate that incorporating CTPG with these approaches significantly enhances performance in manipulation and locomotion benchmarks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 33 references that are in the map. First posted on arXiv 9 Jul 2025.

Builds on

24 of 33 references.

References · 24

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  4. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  6. 2017A Survey on Multi-Task Learningcited by 840
  7. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  8. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  9. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  10. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  11. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 340
  12. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
12 more
  1. 2019Soft Actor-Critic for Discrete Action Settingscited by 174
  2. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  3. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 161
  4. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 143
  5. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  6. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  7. 2017Divide-and-Conquer Reinforcement Learningcited by 48
  8. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 46
  9. 2022Disentangling Transfer in Continual Reinforcement Learningcited by 33
  10. 2017Benchmark Environments for Multitask Learning in Continuous Domainscited by 14
  11. 2023Not All Tasks Are Equally Difficult: Multi-Task Deep Reinforcement Learning with Dynamic Depth Routingcited by 8
  12. 2022CUP: Critic-Guided Policy Reusecited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Knowledge Diversion for Efficient Morphology Control and Policy Transfercited by 1
  2. 2025Leveraging Temporally Extended Behavior Sharing for Multi-task Reinforcement Learningcited by 0
  3. 2026TABX: A High-Throughput Sandbox Battle Simulator for Multi-Agent Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning
  2. 2026A Single Diffusion-Policy Controller for Multi-Task Block Pushing with Zero-Shot Sim-to-Real Transfer
  3. 2024Efficient Multi-Task Reinforcement Learning via Task-Specific Action Correction
  4. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning
  5. 2025Multi-Task Reinforcement Learning Enables Parameter Scaling
  6. 2024On the benefits of pixel-based hierarchical policies for task generalization
  7. 2025Versatile Loco-Manipulation through Flexible Interlimb Coordination
  8. 2024Sharing Knowledge in Multi-Task Deep Reinforcement Learning