mlsanityData to 7 Oct 2026

Beyond Shallow Behavior: Task-Efficient Value-Based Multi-Task Offline MARL via Skill Discovery

arXiv 2502.08985 ↗First posted 13 Feb 2025cs.LG cs.AI cs.MAcited by 1

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

As a data-driven approach, offline MARL learns superior policies solely from offline datasets, ideal for domains rich in historical data but with high interaction costs and risks. However, most existing methods are task-specific, requiring retraining for new tasks, leading to redundancy and inefficiency. To address this issue, we propose a task-efficient value-based multi-task offline MARL algorithm, Skill-Discovery Conservative Q-Learning (SD-CQL). Unlike existing methods decoding actions from skills via behavior cloning, SD-CQL discovers skills in a latent space by reconstructing the next observation, evaluates fixed and variable actions separately, and uses conservative Q-learning with local value calibration to select the optimal action for each skill. It eliminates the need for local-global alignment and enables strong multi-task generalization from limited, small-scale source tasks. Substantial experiments on StarCraft II demonstrate the superior generalization performance and task-efficiency of SD-CQL. It achieves the best performance on 13 out of 14 task sets, with up to 68.9% improvement on individual task sets.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 30 of its 50 references that are in the map. First posted on arXiv 13 Feb 2025.

Builds on

30 of 50 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learning22 Sep
  2. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  3. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  4. 2021Offline Reinforcement Learning as Anti-Exploration11 Jun
  5. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularization14 Mar
  6. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  5. 2019The StarCraft Multi-Agent Challengecited by 589
  6. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  7. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  8. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 281
  9. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 180
  10. 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 143
  11. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  12. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 78
11 more
  1. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  2. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 59
  3. 2019PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learningcited by 46
  4. 2024An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learningcited by 46
  5. 2019From Few to More: Large-scale Dynamic Multiagent Curriculum Learningcited by 36
  6. 2021Offline Decentralized Multi-Agent Reinforcement Learningcited by 32
  7. 2021Regularized Softmax Deep Multi-Agent Q-Learningcited by 19
  8. 2023Maximum Entropy Heterogeneous-Agent Reinforcement Learningcited by 19
  9. 2023Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RLcited by 15
  10. 2023Fast Teammate Adaptation in the Presence of Sudden Policy Changecited by 8
  11. 2024Variational Offline Multi-agent Skill Discoverycited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Multi-Agent Reinforcement Learning via Agent-Specific Preferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Bayesian Nonparametrics for Offline Skill Discovery
  2. 2024Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learning
  3. 2024ACL-QL: Adaptive Conservative Level in Q-Learning for Offline Reinforcement Learning
  4. 2024Robust Policy Learning via Offline Skill Diffusion
  5. 2025Generalisation in Multitask Fitted Q-Iteration and Offline Q-learning
  6. 2022Mildly Conservative Q-Learning for Offline Reinforcement Learning
  7. 2024Strategically Conservative Q-Learning
  8. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning