mlsanityData to 14 Oct 2026

When Skills Remain But Access Fails: Factorized Skill Memory for Continual Skill Internalization

arXiv 2610.15492 ↗First posted 11 Oct 2026cs.AIcited by 0

On the map: LLM agents and tool use → Agent skill learning and evolution

Abstract (arXiv)

Skill internalization aims to convert externally specified procedural skills into persistent model capabilities, allowing language-model agents to execute them without retrieving or reinjecting skill descriptions at inference time. Yet when new skills are learned continually, whether previously internalized skills remain usable is still poorly understood. We study forgetting in continual skill internalization and uncover a clear hierarchical structure. Early in subsequent training, a model may lose the ability to invoke an old skill from the raw request, while supplying only the correct canonical skill name substantially restores the learned behavior; with further optimization, this conditional execution also degrades. We call these two stages skill-addressing forgetting and skill-execution forgetting. Guided by this finding, we introduce Factorized Skill Memory (FSM), which separates continual skill memory into a plastic Address Memory and a stable Execution Memory. At each stage, FSM trains a new LoRA executor using only current-stage data, freezes it after acquisition, and appends it to Execution Memory; in parallel, Address Memory incrementally learns to map current requests to executors without replaying historical requests or execution trajectories. At inference time, Address Memory selects one executor from the raw request, and only that frozen executor performs a single language-model generation. On a five-stage benchmark built from 100 TaskBench skills, FSM achieves 65.95% mean exact match and 93.38% addressing accuracy across two task orders, improving over the strongest replay-free parameter-isolation baseline by 14.3 percentage points while remaining within 1.9 points of Replay, which retains historical training examples.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 30 of its 37 references that are in the map. First posted on arXiv 11 Oct 2026.

Builds on

30 of 37 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Demystifying Agent Skills: Why They Work-Until They Don't14 Aug
  2. 2026Skill-to-LoRA: From Using Skills to Learning Behaviors for Token-Efficient LLM Agents15 Jun
  3. 2026LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents4 Jun
  4. 2026Learning While Acting: A Skill-Enhanced Test-Time Co-Evolution Framework for Online Lifelong Learning Agents3 Jun
  5. 2026SKILLC: Learning Autonomous Skill Internalization in LLM Agents via Contrastive Credit Assignment27 May
  6. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization2 Apr
  7. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks13 Feb

Other references · 23

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  8. 2016Learning without Forgettingcited by 2,496
  9. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  10. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  11. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  12. 2016Progressive Neural Networkscited by 1,361
11 more
  1. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  2. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  3. 2019Three scenarios for continual learningcited by 614
  4. 2017Learning multiple visual domains with residual adapterscited by 557
  5. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
  6. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 312
  7. 2023TaskBench: Benchmarking Large Language Models for Task Automationcited by 144
  8. 2020Continual Learning in Task-Oriented Dialogue Systemscited by 83
  9. 2024Rehearsal-Free Modular and Compositional Continual Learning for Language Modelscited by 27
  10. 2026Parametric Skillscited by 3
  11. 2026PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraftcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ContinualSkillBench: Can LLM Agents Truly Evolve Their Capabilities?
  2. 2026When Not to Imitate: Boundary-Aware Skill Memory for Reliable Tool-Use LLM Agents
  3. 2026When Synthetic Data Hurts: On Catastrophic Forgetting in Skill Retrieval for LLM Agents
  4. 2026SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Tasks
  5. 2026Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agents
  6. 2026Scope Before You Persist: Preventing Cross-Family Interference in Agent Memory
  7. 2026CONTRAMEM: Learning Self-Evolving Procedural Memory from Contrasting Multi-Model Trajectories
  8. 2026SkillApt: Learning When to Activate Agent Skills from Counterfactual Evidence