mlsanityData to 14 Oct 2026

Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learning

arXiv 2507.06628 ↗First posted 9 Jul 2025cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Offline multi-task reinforcement learning aims to learn a unified policy capable of solving multiple tasks using only pre-collected task-mixed datasets, without requiring any online interaction with the environment. However, it faces significant challenges in effectively sharing knowledge across tasks. Inspired by the efficient knowledge abstraction observed in human learning, we propose Goal-Oriented Skill Abstraction (GO-Skill), a novel approach designed to extract and utilize reusable skills to enhance knowledge transfer and task performance. Our approach uncovers reusable skills through a goal-oriented skill extraction process and leverages vector quantization to construct a discrete skill library. To mitigate class imbalances between broadly applicable and task-specific skills, we introduce a skill enhancement phase to refine the extracted skills. Furthermore, we integrate these skills using hierarchical policy learning, enabling the construction of a high-level policy that dynamically orchestrates discrete skills to accomplish specific tasks. Extensive experiments on diverse robotic manipulation tasks within the MetaWorld benchmark demonstrate the effectiveness and versatility of GO-Skill.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 38 of its 48 references that are in the map. First posted on arXiv 9 Jul 2025.

Builds on

38 of 48 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020End-to-End Object Detection with Transformerscited by 7,414
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2015Continuous control with deep reinforcement learningcited by 3,407
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  8. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  9. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  10. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  11. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  12. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
26 more
  1. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  2. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  3. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  4. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  5. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  6. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  7. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  8. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  9. 2022Multi-Game Decision Transformerscited by 214
  10. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 188
  11. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 182
  12. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  13. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 161
  14. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 146
  15. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 143
  16. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 115
  17. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RLcited by 103
  18. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  19. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  20. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 76
  21. 2023Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasetscited by 58
  22. 2022Offline RL Policies Should be Trained to be Adaptivecited by 52
  23. 2024Sharing Knowledge in Multi-Task Deep Reinforcement Learningcited by 44
  24. 2021Hierarchical Few-Shot Imitation with Skill Transition Modelscited by 37
  25. 2020An Empirical Study of Representation Learning for Reinforcement Learning in Healthcarecited by 34
  26. 2023Not All Tasks Are Equally Difficult: Multi-Task Deep Reinforcement Learning with Dynamic Depth Routingcited by 8

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Soft Conflict-Resolution Decision Transformer for Offline Multi-Task Reinforcement Learningcited by 2
  2. 2025Generalisation in Multitask Fitted Q-Iteration and Offline Q-learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Data
  2. 2024Robust Policy Learning via Offline Skill Diffusion
  3. 2024Offline Policy Learning via Skill-step Abstraction for Long-horizon Goal-Conditioned Tasks
  4. 2024Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learning
  5. 2023Offline Skill Generalization via Task and Motion Planning
  6. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learning
  7. 2026Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policies
  8. 2025Knowledge capture, adaptation and composition (KCAC): A framework for cross-task curriculum learning in robotic manipulation