mlsanityData to 7 Oct 2026

Contrastive Modules with Temporal Attention for Multi-Task Reinforcement Learning

arXiv 2311.01075 ↗First posted 2 Nov 2023cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

In the field of multi-task reinforcement learning, the modular principle, which involves specializing functionalities into different modules and combining them appropriately, has been widely adopted as a promising approach to prevent the negative transfer problem that performance degradation due to conflicts between tasks. However, most of the existing multi-task RL methods only combine shared modules at the task level, ignoring that there may be conflicts within the task. In addition, these methods do not take into account that without constraints, some modules may learn similar functions, resulting in restricting the model's expressiveness and generalization capability of modular methods. In this paper, we propose the Contrastive Modules with Temporal Attention(CMTA) method to address these limitations. CMTA constrains the modules to be different from each other by contrastive learning and combining shared modules at a finer granularity than the task level with temporal attention, alleviating the negative transfer within the task and improving the generalization ability and the performance for multi-task RL. We conducted the experiment on Meta-World, a multi-task RL benchmark containing various robotics manipulation tasks. Experimental results show that CMTA outperforms learning each task individually for the first time and achieves substantial performance improvements over the baselines.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 2 Nov 2023.

Builds on

36 of 49 references.

References · 36

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  6. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  7. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  8. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  9. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  10. 2018End-to-End Multi-Task Learning with Attentioncited by 533
  11. 2016Cross-stitch Networks for Multi-task Learningcited by 466
  12. 2015Neural Module Networkscited by 375
24 more
  1. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 374
  2. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  3. 2013Learning Factored Representations in a Deep Mixture of Expertscited by 275
  4. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  5. 2019Pareto Multi-Task Learningcited by 224
  6. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  7. 2019Task2Vec: Task Embedding for Meta-Learningcited by 197
  8. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  9. 2016Recurrent Neural Network for Text Classification with Multi-Task Learningcited by 144
  10. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  11. 2017Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learningcited by 125
  12. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
  13. 2020Understanding and Improving Information Transfer in Multi-Task Learningcited by 113
  14. 2017Learning Independent Causal Mechanismscited by 104
  15. 2016HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognitioncited by 103
  16. 2018Task-Embedded Control Networks for Few-Shot Imitation Learningcited by 65
  17. 2017Identifying beneficial task relations for multi-task learning in deep neural networkscited by 59
  18. 2016Learning to Push by Grasping: Using multiple tasks for effective learningcited by 36
  19. 2022Causality-driven Hierarchical Structure Discovery for Reinforcement Learningcited by 23
  20. 2023Self-driven Grounding: Large Language Model Agents with Automatical Language-aligned Skill Learningcited by 9
  21. 2023Conceptual Reinforcement Learning for Language-Conditioned Taskscited by 8
  22. 2021Hindsight Value Function for Variance Reduction in Stochastic Dynamic Environmentcited by 7
  23. 2022Object-Category Aware Reinforcement Learningcited by 6
  24. 2023Online Prototype Alignment for Few-shot Policy Transfercited by 4

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 5.

8 papers.

  1. 2024Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignmentcited by 7
  2. 2024Offline Reinforcement Learning from Datasets with Structured Non-Stationaritycited by 5
  3. 2025DETACH: Cross-domain Learning for Long-Horizon Tasks via Mixture of Disentangled Expertscited by 5
  4. 2025Learning Policy Committees for Effective Personalization in MDPs with Diverse Taskscited by 4
  5. 2025CoMoE: Contrastive Representation for Mixture-of-Experts in Parameter-Efficient Fine-tuningcited by 4
  6. 2025One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planningcited by 3
  7. 2024Centralized Reward Agent for Knowledge Sharing and Transfer in Multi-Task Reinforcement Learningcited by 0
  8. 2025Complex Instruction Following with Diverse Style Policies in Football Gamescited by 0