mlsanityData to 7 Oct 2026

Skill Expansion and Composition in Parameter Space

arXiv 2502.05932 ↗First posted 9 Feb 2025cs.LG cs.AI cs.ROcited by 13

Not on the map. Why?

Abstract (arXiv)

Humans excel at reusing prior knowledge to address new challenges and developing skills while solving problems. This paradigm becomes increasingly popular in the development of autonomous agents, as it develops systems that can self-evolve in response to new challenges like human beings. However, previous methods suffer from limited training efficiency when expanding new skills and fail to fully leverage prior knowledge to facilitate new task learning. In this paper, we propose Parametric Skill Expansion and Composition (PSEC), a new framework designed to iteratively evolve the agents' capabilities and efficiently address new challenges by maintaining a manageable skill library. This library can progressively integrate skill primitives as plug-and-play Low-Rank Adaptation (LoRA) modules in parameter-efficient finetuning, facilitating efficient and flexible skill expansion. This structure also enables the direct skill compositions in parameter space by merging LoRA modules that encode different skills, leveraging shared information across skills to effectively program new skills. Based on this, we propose a context-aware module to dynamically activate different skills to collaboratively handle new tasks. Empowering diverse applications including multi-objective composition, dynamics shift, and continual policy shift, the results on D4RL, DSRL benchmarks, and the DeepMind Control Suite show that PSEC exhibits superior capacity to leverage prior knowledge to efficiently tackle new challenges, as well as expand its skill libraries to evolve the capabilities. Project website: https://ltlhuuu.github.io/PSEC/.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 88 of its 125 references that are in the map. First posted on arXiv 9 Feb 2025.

Builds on

88 of 125 references.

References · 88

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  6. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  8. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  9. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  10. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  11. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  12. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
38 more
  1. 2018Experience Replay for Continual Learningcited by 1,074
  2. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  3. 2018DeepMind Control Suitecited by 935
  4. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  6. 2019Solving Rubik's Cube with a Robot Handcited by 699
  7. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  8. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  9. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  10. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  11. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  12. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  13. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  14. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  15. 2015Neural Module Networkscited by 375
  16. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 359
  17. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
  18. 2024Diffusion Policy Policy Optimizationcited by 255
  19. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  20. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMCcited by 216
  21. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 211
  22. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 177
  23. 2023Composing Parameter-Efficient Modules with Arithmetic Operationscited by 155
  24. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 145
  25. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  26. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 138
  27. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
  28. 2023Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularizationcited by 114
  29. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 112
  30. 2018Meta-Learning Probabilistic Inference For Predictioncited by 108
  31. 2019MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policiescited by 105
  32. 2020Continual Deep Learning by Functional Regularisation of Memorable Pastcited by 104
  33. 2023Policy Expansion for Bridging Offline-to-Online Reinforcement Learningcited by 103
  34. 2018Neural probabilistic motor primitives for humanoid controlcited by 96
  35. 2023Compositional Foundation Models for Hierarchical Planningcited by 92
  36. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 90
  37. 2019Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvementcited by 88
  38. 2024Multi-LoRA Composition for Image Generationcited by 80

50 most cited of 88 All 88 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 7.

13 papers.

Sort:

By area:

  1. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
  2. 2025Decoupled Diffusion Sparks Adaptive Scene Generationcited by 12
  3. 2026Unleashing the Potential of Diffusion Models for End-to-End Autonomous Drivingcited by 11
  4. 2025Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challengescited by 3
  5. 2026Learning Diffusion Policy from Primitive Skills for Robot Manipulationcited by 3
  6. 2025Efficient Cross-Domain Offline Reinforcement Learning with Dynamics- and Value-Aligned Data Filteringcited by 2
  7. 2025Diffusion Policies with Value-Conditional Optimization for Offline Reinforcement Learningcited by 1
  8. 2026DADP: Domain Adaptive Diffusion Policycited by 1
  9. 2026Cross-Domain Offline Policy Adaptation via Selective Transition Correctioncited by 1
  10. 2026Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Trainingcited by 1
  11. 2025Multimodal LLMs Do Not Compose Skills Optimally Across Modalitiescited by 0
  12. 2026Continual Quadruped Robots Coordination via Semantic Skill Discoverycited by 0
1 more
  1. 2026SAMBAR: Selective Anchoring via Method of Multipliers for Balanced Knowledge Acquisition and Retention in Vision-Language-Action Modelscited by 0