mlsanityData to 7 Oct 2026

PaCo: Parameter-Compositional Multi-Task Reinforcement Learning

arXiv 2210.11653 ↗First posted 21 Oct 2022cs.LG cs.AI cs.ROcited by 40

Not on the map. Why?

Abstract (arXiv)

The purpose of multi-task reinforcement learning (MTRL) is to train a single policy that can be applied to a set of different tasks. Sharing parameters allows us to take advantage of the similarities among tasks. However, the gaps between contents and difficulties of different tasks bring us challenges on both which tasks should share the parameters and what parameters should be shared, as well as the optimization challenges due to parameter sharing. In this work, we introduce a parameter-compositional approach (PaCo) as an attempt to address these challenges. In this framework, a policy subspace represented by a set of parameters is learned. Policies for all the single tasks lie in this subspace and can be composed by interpolating with the learned set. It allows not only flexible parameter sharing but also a natural way to improve training. We demonstrate the state-of-the-art performance on Meta-World benchmarks, verifying the effectiveness of the proposed approach.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

27 of 42 references.

References · 27

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  3. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  4. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  6. 2017A Survey on Multi-Task Learningcited by 838
  7. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  8. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  9. 2017Federated Multi-Task Learningcited by 669
  10. 2018End-to-End Multi-Task Learning with Attentioncited by 533
  11. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  12. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
15 more
  1. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  2. 2019Pareto Multi-Task Learningcited by 224
  3. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  4. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  5. 2020Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Modelscited by 167
  6. 2017Parameter Space Noise for Explorationcited by 158
  7. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 138
  8. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  9. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
  10. 2018Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learningcited by 52
  11. 2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 33
  12. 2020Knowledge Transfer in Multi-Task Deep Reinforcement Learning for Continuous Controlcited by 27
  13. 2021Meta-learning with an Adaptive Task Schedulercited by 27
  14. 2018Multi-task Learning with Sample Re-weighting for Machine Reading Comprehensioncited by 8
  15. 2018Multi-task Learning for Continuous Controlcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 10; 2025: 16; 2026: 7.

40 papers.

Sort:

By area:

  1. 2025Interactive Post-Training for Vision-Language-Action Modelscited by 129
  2. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  3. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  4. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learningcited by 56
  5. 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 45
  6. 2025Meta-World+: An Improved, Standardized, RL Benchmarkcited by 37
  7. 2023Interactive Planning Using Large Language Models for Partially Observable Robotics Taskscited by 27
  8. 2024Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Trainingcited by 26
  9. 2025Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learnerscited by 23
  10. 2025Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformercited by 15
  11. 2024HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 14
  12. 2025Skill Expansion and Composition in Parameter Spacecited by 13
28 more
  1. 2023Task Generalization with Stability Guarantees via Elastic Dynamical System Motion Policiescited by 10
  2. 2023Human-oriented Representation Learning for Robotic Manipulationcited by 10
  3. 2023Not All Tasks Are Equally Difficult: Multi-Task Deep Reinforcement Learning with Dynamic Depth Routingcited by 8
  4. 2024Active Fine-Tuning of Multi-Task Policiescited by 8
  5. 2025Multi-Task Reinforcement Learning Enables Parameter Scalingcited by 8
  6. 2024Model-Based Transfer Learning for Contextual Reinforcement Learningcited by 7
  7. 2023Efficient Multi-Task and Transfer Reinforcement Learning with Parameter-Compositional Frameworkcited by 6
  8. 2025Scalable Multi-Objective and Meta Reinforcement Learning via Gradient Estimationcited by 6
  9. 2025SLIM: Sim-to-Real Legged Instructive Manipulation via Long-Horizon Visuomotor Learningcited by 4
  10. 2025Learning Policy Committees for Effective Personalization in MDPs with Diverse Taskscited by 4
  11. 2025Plasticity-Aware Mixture of Experts for Learning Under QoE Shifts in Adaptive Video Streamingcited by 4
  12. 2025Efficient Multi-Task Reinforcement Learning with Cross-Task Policy Guidancecited by 3
  13. 2025One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planningcited by 3
  14. 2025Sparsity-Driven Plasticity in Multi-Task Reinforcement Learningcited by 2
  15. 2025Soft Conflict-Resolution Decision Transformer for Offline Multi-Task Reinforcement Learningcited by 2
  16. 2026Structure Detection for Contextual Reinforcement Learningcited by 2
  17. 2024Task-Agnostic Pre-training and Task-Guided Fine-tuning for Versatile Diffusion Plannercited by 1
  18. 2024Task-Aware Harmony Multi-Task Decision Transformer for Offline Reinforcement Learningcited by 1
  19. 2026Continual Policy Consolidation for Lifelong Robot Learningcited by 1
  20. 2024Shared-unique Features and Task-aware Prioritized Sampling on Multi-task Reinforcement Learningcited by 0
  21. 2024Centralized Reward Agent for Knowledge Sharing and Transfer in Multi-Task Reinforcement Learningcited by 0
  22. 2025Policy Search, Retrieval, and Composition via Task Similarity in Collaborative Agentic Systemscited by 0
  23. 2025Leveraging Temporally Extended Behavior Sharing for Multi-task Reinforcement Learningcited by 0
  24. 2026TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancingcited by 0
  25. 2026Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Samplingcited by 0
  26. 2026CDCP: Conditional Diffusion Model with Contextual Prompts for Multi-task Offline Safe Reinforcement Learningcited by 0
  27. 2026Task Specialization Fine-Tuning for Contextual Reinforcement Learningcited by 0
  28. 2026From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Interventioncited by 0