mlsanityData to 7 Oct 2026

Multi-Task Reinforcement Learning with Soft Modularization

arXiv 2003.13661 ↗First posted 30 Mar 2020cs.LG cs.AI cs.ROcited by 138

Not on the map. Why?

Abstract (arXiv)

Multi-task learning is a very challenging problem in reinforcement learning. While training multiple tasks jointly allow the policies to share parameters across different tasks, the optimization problem becomes non-trivial: It remains unclear what parameters in the network should be reused across tasks, and how the gradients from different tasks may interfere with each other. Thus, instead of naively sharing parameters across tasks, we introduce an explicit modularization technique on policy representation to alleviate this optimization issue. Given a base policy network, we design a routing network which estimates different routing strategies to reconfigure the base network for each task. Instead of directly selecting routes for each task, our task-specific policy uses a method called soft modularization to softly combine all the possible routes, which makes it suitable for sequential tasks. We experiment with various robotics manipulation tasks in simulation and show our method improves both sample efficiency and performance over strong baselines by a large margin.

Key facts. Cited by 138 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 24; 2023: 29; 2024: 27; 2025: 25; 2026: 16.

138 papers.

Sort:

By area:

  1. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  2. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  3. 2022Multi-Task Learning as a Bargaining Gamecited by 209
  4. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  5. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  6. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  7. 2023GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fieldscited by 99
  8. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  9. 2023FAMO: Fast Adaptive Multitask Optimizationcited by 74
  10. 2023Independent Component Alignment for Multi-Task Learningcited by 69
  11. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  12. 2023Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learningcited by 50
38 more
  1. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 45
  2. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 40
  3. 2022Modular Lifelong Reinforcement Learning via Neural Compositioncited by 39
  4. 2022The Neural Race Reduction: Dynamics of Abstraction in Gated Networkscited by 37
  5. 2025Meta-World+: An Improved, Standardized, RL Benchmarkcited by 37
  6. 2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
  7. 2023Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithmscited by 33
  8. 2024LLaMAR: Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environmentscited by 32
  9. 2022Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learningcited by 28
  10. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  11. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  12. 2024Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Trainingcited by 26
  13. 2021Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problemscited by 25
  14. 2021Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environmentscited by 24
  15. 2024DNAct: Diffusion Guided Multi-Task 3D Policy Learningcited by 22
  16. 2022CompoSuite: A Compositional Reinforcement Learning Benchmarkcited by 21
  17. 2023Universal Morphology Control via Contextual Modulationcited by 21
  18. 2021The Functional Correspondence Problemcited by 20
  19. 2024Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agentcited by 20
  20. 2023Large Sequence Models for Sequential Decision-Making: A Surveycited by 19
  21. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 18
  22. 2022Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Taskscited by 17
  23. 2022Robust Policy Learning over Multiple Uncertainty Setscited by 16
  24. 2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
  25. 2024Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulationcited by 15
  26. 2025Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformercited by 15
  27. 2022Multi-Agent Policy Transfer via Task Relationship Modelingcited by 14
  28. 2022Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approachcited by 14
  29. 2024HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 14
  30. 2024Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretrainingcited by 13
  31. 2025Skill Expansion and Composition in Parameter Spacecited by 13
  32. 2022A System for Morphology-Task Generalization via Unified Representation and Behavior Distillationcited by 12
  33. 2024Make-An-Agent: A Generalizable Policy Network Generator with Behavior-Prompted Diffusioncited by 12
  34. 2024Multi-Task Reinforcement Learning for Quadrotorscited by 12
  35. 2022Beyond Supervised Continual Learning: a Reviewcited by 11
  36. 2022Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challengescited by 10
  37. 2022Transporters with Visual Foresight for Solving Unseen Rearrangement Taskscited by 9
  38. 2023Improvable Gap Balancing for Multi-Task Learningcited by 9

50 most cited of 138 All 138 (JSON)