# Skill Expansion and Composition in Parameter Space

arXiv: https://arxiv.org/abs/2502.05932 · HTML: https://mlsanity.com/paper/2502.05932/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 88 of its 125 references that are in the map.
- First posted on arXiv 9 Feb 2025.

## Builds on (most cited first, 30 of 88)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [AWAC: Accelerating Online Reinforcement Learning with Offline Datasets](https://mlsanity.com/paper/2006.09359/) (2020)
- [MOPO: Model-based Offline Policy Optimization](https://mlsanity.com/paper/2005.13239/) (2020)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning](https://mlsanity.com/paper/2208.06193/) (2022)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Is Conditional Generative Modeling all you need for Decision-Making?](https://mlsanity.com/paper/2211.15657/) (2022)
- [Conflict-Averse Gradient Descent for Multi-task Learning](https://mlsanity.com/paper/2110.14048/) (2021)
- [Directly Fine-Tuning Diffusion Models on Differentiable Rewards](https://mlsanity.com/paper/2309.17400/) (2023)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition](https://mlsanity.com/paper/2307.13269/) (2023)
- [IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies](https://mlsanity.com/paper/2304.10573/) (2023)
- [Diffusion Policy Policy Optimization](https://mlsanity.com/paper/2409.00588/) (2024)

## Built on by (most cited first, 14 of 14)

- [X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model](https://mlsanity.com/paper/2510.10274/) (2025)
- [Decoupled Diffusion Sparks Adaptive Scene Generation](https://mlsanity.com/paper/2504.10485/) (2025)
- [Unleashing the Potential of Diffusion Models for End-to-End Autonomous Driving](https://mlsanity.com/paper/2602.22801/) (2026)
- [Learning Diffusion Policy from Primitive Skills for Robot Manipulation](https://mlsanity.com/paper/2601.01948/) (2026)
- [Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges](https://mlsanity.com/paper/2508.17449/) (2025)
- [Efficient Cross-Domain Offline Reinforcement Learning with Dynamics- and Value-Aligned Data Filtering](https://mlsanity.com/paper/2512.02435/) (2025)
- [Diffusion Policies with Value-Conditional Optimization for Offline Reinforcement Learning](https://mlsanity.com/paper/2511.08922/) (2025)
- [DADP: Domain Adaptive Diffusion Policy](https://mlsanity.com/paper/2602.04037/) (2026)
- [Cross-Domain Offline Policy Adaptation via Selective Transition Correction](https://mlsanity.com/paper/2602.05776/) (2026)
- [Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Training](https://mlsanity.com/paper/2607.19971/) (2026)
- [Multimodal LLMs Do Not Compose Skills Optimally Across Modalities](https://mlsanity.com/paper/2511.08113/) (2025)
- [Continual Quadruped Robots Coordination via Semantic Skill Discovery](https://mlsanity.com/paper/2606.08102/) (2026)
- [SAMBAR: Selective Anchoring via Method of Multipliers for Balanced Knowledge Acquisition and Retention in Vision-Language-Action Models](https://mlsanity.com/paper/2609.32108/) (2026)
- [SkillFusion: Towards Reliable Multi-Skill Composition for LLM Agents](https://mlsanity.com/paper/2610.15481/) (2026)

## Abstract (arXiv)

Humans excel at reusing prior knowledge to address new challenges and developing skills while solving problems. This paradigm becomes increasingly popular in the development of autonomous agents, as it develops systems that can self-evolve in response to new challenges like human beings. However, previous methods suffer from limited training efficiency when expanding new skills and fail to fully leverage prior knowledge to facilitate new task learning. In this paper, we propose Parametric Skill Expansion and Composition (PSEC), a new framework designed to iteratively evolve the agents' capabilities and efficiently address new challenges by maintaining a manageable skill library. This library can progressively integrate skill primitives as plug-and-play Low-Rank Adaptation (LoRA) modules in parameter-efficient finetuning, facilitating efficient and flexible skill expansion. This structure also enables the direct skill compositions in parameter space by merging LoRA modules that encode different skills, leveraging shared information across skills to effectively program new skills. Based on this, we propose a context-aware module to dynamically activate different skills to collaboratively handle new tasks. Empowering diverse applications including multi-objective composition, dynamics shift, and continual policy shift, the results on D4RL, DSRL benchmarks, and the DeepMind Control Suite show that PSEC exhibits superior capacity to leverage prior knowledge to efficiently tackle new challenges, as well as expand its skill libraries to evolve the capabilities. Project website: https://ltlhuuu.github.io/PSEC/.