# Lifelong Robot Library Learning: Bootstrapping Composable and Generalizable Skills for Embodied Control with Language Models

arXiv: https://arxiv.org/abs/2406.18746 · HTML: https://mlsanity.com/paper/2406.18746/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2024.

## Built on by (most cited first, 27 of 27)

- [ASPIRE: Agentic /Skills Discovery for Robotics](https://mlsanity.com/paper/2607.00272/) (2026)
- [A Survey of Continual Reinforcement Learning](https://mlsanity.com/paper/2506.21872/) (2025)
- [Continual Learning for Generative AI: From LLMs to MLLMs and Beyond](https://mlsanity.com/paper/2506.13045/) (2025)
- [The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement](https://mlsanity.com/paper/2609.11873/) (2026)
- [SPECI: Skill Prompts based Hierarchical Continual Imitation Learning for Robot Manipulation](https://mlsanity.com/paper/2504.15561/) (2025)
- [Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Braking](https://mlsanity.com/paper/2410.08616/) (2024)
- [Memory Transfer Planning: LLM-driven Context-Aware Code Adaptation for Robot Manipulation](https://mlsanity.com/paper/2509.24160/) (2025)
- [Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents](https://mlsanity.com/paper/2610.02204/) (2026)
- [ViReSkill: Vision-Grounded Replanning with Skill Memory for LLM-Based Planning in Lifelong Robot Learning](https://mlsanity.com/paper/2509.24219/) (2025)
- [Learning and Transferring Closed-Loop Robot Software](https://mlsanity.com/paper/2609.19906/) (2026)
- [RAG-Modulo: Solving Sequential Tasks using Experience, Critics, and Language Models](https://mlsanity.com/paper/2409.12294/) (2024)
- [Task-agnostic Lifelong Robot Learning with Retrieval-based Weighted Local Adaptation](https://mlsanity.com/paper/2410.02995/) (2024)
- [Lifelong Reinforcement Learning with Similarity-Driven Weighting by Large Models](https://mlsanity.com/paper/2503.12923/) (2025)
- [OpenGo: An OpenClaw-Based Robotic Dog with Real-Time Skill Switching](https://mlsanity.com/paper/2604.01708/) (2026)
- [Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills](https://mlsanity.com/paper/2608.01851/) (2026)
- [SkillComposer: Learning Reusable Skills for Natural-Language Robot Programming](https://mlsanity.com/paper/2608.14944/) (2026)
- [Teach and Grow: An Agent-Centered Architecture for General Robot Learning](https://mlsanity.com/paper/2608.17209/) (2026)
- [SimSkill: A Self-Evolving LLM Agent for Skill and Knowledge Accumulation in Traffic Simulation](https://mlsanity.com/paper/2609.03753/) (2026)
- [In-Context Learning for Robots: Methods and Applications](https://mlsanity.com/paper/2609.36012/) (2026)
- [EnvBridge: Bridging Diverse Environments with Cross-Environment Knowledge Transfer for Embodied AI](https://mlsanity.com/paper/2410.16919/) (2024)
- [Efficient Skill Grounding via Code Refactoring with Small Language Models](https://mlsanity.com/paper/2606.07999/) (2026)
- [CLASP: Language-Driven Robot Skill Selection and Composition using Task-Parameterized Learning](https://mlsanity.com/paper/2606.08169/) (2026)
- [Functional Cache Grafting: Robust and Rapid Code-Policy Synthesis for Embodied Agents](https://mlsanity.com/paper/2606.13097/) (2026)
- [Evolving in the Agent Jungle via History-Informed Opponent Awareness](https://mlsanity.com/paper/2608.02005/) (2026)
- [Towards a Systems Foundation for Agentic Skills: Architecture, Lifecycle, and Security](https://mlsanity.com/paper/2608.29596/) (2026)
- [REFACTOR-VLA: Unsupervised Library Learning of Typed Motor Programs](https://mlsanity.com/paper/2609.01215/) (2026)
- [Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnesses](https://mlsanity.com/paper/2609.36630/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have emerged as a new paradigm for embodied reasoning and control, most recently by generating robot policy code that utilizes a custom library of vision and control primitive skills. However, prior arts fix their skills library and steer the LLM with carefully hand-crafted prompt engineering, limiting the agent to a stationary range of addressable tasks. In this work, we introduce LRLL, an LLM-based lifelong learning agent that continuously grows the robot skill library to tackle manipulation tasks of ever-growing complexity. LRLL achieves this with four novel contributions: 1) a soft memory module that allows dynamic storage and retrieval of past experiences to serve as context, 2) a self-guided exploration policy that proposes new tasks in simulation, 3) a skill abstractor that distills recent experiences into new library skills, and 4) a lifelong learning algorithm for enabling human users to bootstrap new skills with minimal online interaction. LRLL continuously transfers knowledge from the memory to the library, building composable, general and interpretable policies, while bypassing gradient-based optimization, thus relieving the learner from catastrophic forgetting. Empirical evaluation in a simulated tabletop environment shows that LRLL outperforms end-to-end and vanilla LLM approaches in the lifelong setup while learning skills that are transferable to the real world. Project material will become available at the webpage https://gtziafas.github.io/LRLL_project.