mlsanityData to 7 Oct 2026

Soft Conflict-Resolution Decision Transformer for Offline Multi-Task Reinforcement Learning

arXiv 2511.13133 ↗First posted 17 Nov 2025cs.LG cs.AIcited by 2

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Multi-task reinforcement learning (MTRL) seeks to learn a unified policy for diverse tasks, but often suffers from gradient conflicts across tasks. Existing masking-based methods attempt to mitigate such conflicts by assigning task-specific parameter masks. However, our empirical study shows that coarse-grained binary masks have the problem of over-suppressing key conflicting parameters, hindering knowledge sharing across tasks. Moreover, different tasks exhibit varying conflict levels, yet existing methods use a one-size-fits-all fixed sparsity strategy to keep training stability and performance, which proves inadequate. These limitations hinder the model's generalization and learning efficiency. To address these issues, we propose SoCo-DT, a Soft Conflict-resolution method based by parameter importance. By leveraging Fisher information, mask values are dynamically adjusted to retain important parameters while suppressing conflicting ones. In addition, we introduce a dynamic sparsity adjustment strategy based on the Interquartile Range (IQR), which constructs task-specific thresholding schemes using the distribution of conflict and harmony scores during training. To enable adaptive sparsity evolution throughout training, we further incorporate an asymmetric cosine annealing schedule to continuously update the threshold. Experimental results on the Meta-World benchmark show that SoCo-DT outperforms the state-of-the-art method by 7.6% on MT50 and by 10.5% on the suboptimal dataset, demonstrating its effectiveness in mitigating gradient conflicts and improving overall multi-task performance.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 21 of its 27 references that are in the map. First posted on arXiv 17 Nov 2025.

Builds on

21 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Prompting Decision Transformer for Few-Shot Policy Generalization27 Jun
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  6. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  8. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  9. 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 199
  10. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  11. 2020Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Modelscited by 167
  12. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
7 more
  1. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  2. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 68
  3. 2022PaCo: Parameter-Compositional Multi-Task Reinforcement Learningcited by 40
  4. 2023Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusioncited by 36
  5. 2019Learning Sparse Sharing Architectures for Multiple Taskscited by 31
  6. 2021Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancementcited by 28
  7. 2025Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Advancing General-Purpose Reasoning Models with Modular Gradient Surgerycited by 9
  2. 2026HoRD: Robust Humanoid Control via History-Conditioned Reinforcement Learning and Online Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Task-Aware Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning
  2. 2025Sparsity-Driven Plasticity in Multi-Task Reinforcement Learning
  3. 2024HarmoDT: Harmony Multi-Task Decision Transformer for Offline Reinforcement Learning
  4. 2025Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer
  5. 2026SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs
  6. 2023Context Shift Reduction for Offline Meta-Reinforcement Learning
  7. 2026GD2PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimization
  8. 2024Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learning