mlsanityData to 7 Oct 2026

Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learning

arXiv 2502.03752 ↗First posted 6 Feb 2025cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Meta-reinforcement learning (Meta-RL) facilitates rapid adaptation to unseen tasks but faces challenges in long-horizon environments. Skill-based approaches tackle this by decomposing state-action sequences into reusable skills and employing hierarchical decision-making. However, these methods are highly susceptible to noisy offline demonstrations, leading to unstable skill learning and degraded performance. To address this, we propose Self-Improving Skill Learning (SISL), which performs self-guided skill refinement using decoupled high-level and skill improvement policies, while applying skill prioritization via maximum return relabeling to focus updates on task-relevant trajectories, resulting in robust and stable adaptation even under noisy and suboptimal data. By mitigating the effect of noise, SISL achieves reliable skill learning and consistently outperforms other skill-based meta-RL methods on diverse long-horizon tasks. Our code is available at https://epsilog.github.io/SISL.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 57 of its 77 references that are in the map. First posted on arXiv 6 Feb 2025.

Builds on

57 of 77 references.

References · 57

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  5. 2018Exploration by Random Network Distillationcited by 966
  6. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  7. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  8. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  9. 2016The Option-Critic Architecturecited by 433
  10. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  11. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  12. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
38 more
  1. 2016Variational Intrinsic Controlcited by 262
  2. 2017Noisy Networks for Explorationcited by 262
  3. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  4. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
  5. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  6. 2018Variational Option Discovery Algorithmscited by 121
  7. 2017Meta Learning Shared Hierarchiescited by 93
  8. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 73
  9. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 65
  10. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
  11. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 52
  12. 2022Skill-based Model-based Reinforcement Learningcited by 50
  13. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  14. 2018Unsupervised Meta-Learning for Reinforcement Learningcited by 38
  15. 2019Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewardscited by 38
  16. 2021Learning more skills through optimistic explorationcited by 37
  17. 2021Hierarchical Skills for Efficient Explorationcited by 37
  18. 2022Skill-based Meta-Reinforcement Learningcited by 36
  19. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 33
  20. 2018Learning Abstract Optionscited by 31
  21. 2021A Max-Min Entropy Framework for Reinforcement Learningcited by 28
  22. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  23. 2020Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabelingcited by 24
  24. 2021The Logical Options Frameworkcited by 24
  25. 2022Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Roboticscited by 23
  26. 2020Diversity Actor-Critic: Sample-Aware Entropy Regularization for Sample-Efficient Explorationcited by 20
  27. 2023Sample-Efficient and Safe Deep Reinforcement Learning via Reset Deep Ensemble Agentscited by 12
  28. 2023FoX: Formation-aware exploration in multi-agent reinforcement learningcited by 9
  29. 2023Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitationscited by 9
  30. 2024Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learningcited by 9
  31. 2021Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RLcited by 8
  32. 2022Meta-Learning Parameterized Skillscited by 7
  33. 2022ASPiRe:Adaptive Skill Priors for Reinforcement Learningcited by 7
  34. 2023ReRoGCRL: Representation-based Robustness in Goal-Conditioned Reinforcement Learningcited by 6
  35. 2024Hierarchical Transformers are Efficient Meta-Reinforcement Learnerscited by 6
  36. 2025Wolfpack Adversarial Attack for Robust Multi-Agent Reinforcement Learningcited by 6
  37. 2021Provable Hierarchy-Based Meta-Reinforcement Learningcited by 5
  38. 2021Hindsight Foresight Relabeling for Meta-Reinforcement Learningcited by 4

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Retaining Suboptimal Actions to Follow Shifting Optima in Multi-Agent Reinforcement Learningcited by 3
  2. 2025Strict Subgoal Execution: Reliable Long-Horizon Planning in Hierarchical Reinforcement Learningcited by 1
  3. 2026Interaction-Breaking Adversarial Learning Framework for Robust Multi-Agent Reinforcement Learningcited by 1
  4. 2026Bridging Domain Gaps with Target-Aligned Generation for Offline Reinforcement Learningcited by 0
  5. 2026Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learningcited by 0