mlsanityData to 7 Oct 2026

Strategy Discovery and Mixture in Lifelong Learning from Heterogeneous Demonstration

arXiv 2202.07014 ↗First posted 14 Feb 2022cs.LGcited by 1

On the map: Offline RL and imitation learning from data → Robot learning from demonstration

Abstract (arXiv)

Learning from Demonstration (LfD) approaches empower end-users to teach robots novel tasks via demonstrations of the desired behaviors, democratizing access to robotics. A key challenge in LfD research is that users tend to provide heterogeneous demonstrations for the same task due to various strategies and preferences. Therefore, it is essential to develop LfD algorithms that ensure textitflexibility (the robot adapts to personalized strategies), textitefficiency (the robot achieves sample-efficient adaptation), and textitscalability (robot reuses a concise set of strategies to represent a large amount of behaviors). In this paper, we propose a novel algorithm, Dynamic Multi-Strategy Reward Distillation (DMSRD), which distills common knowledge between heterogeneous demonstrations, leverages learned strategies to construct mixture policies, and continues to improve by learning from all available data. Our personalized, federated, and lifelong LfD architecture surpasses benchmarks in two continuous control problems with an average 77% improvement in policy returns and 42% improvement in log likelihood, alongside stronger task reward correlation and more precise strategy rewards.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 37 references that are in the map. First posted on arXiv 14 Feb 2022.

Builds on

17 of 37 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Towards Sample-efficient Apprenticeship Learning from Suboptimal Demonstration8 Oct
  2. 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regression17 Oct

Other references · 15

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2016OpenAI Gymcited by 1,684
  3. 2016Concrete Problems in AI Safetycited by 1,582
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  7. 2019Continual Learning for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challengescited by 92
  8. 2019Learning Reward Functions by Integrating Human Demonstrations and Preferencescited by 64
  9. 2014Efficient Model Learning for Human-Robot Collaborative Taskscited by 35
  10. 2020Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillationcited by 19
  11. 2019Interpretable and Personalized Apprenticeship Scheduling: Learning Interpretable Scheduling Policies from Heterogeneous User Demonstrationscited by 14
  12. 2022Iterated Reasoning with Mutual Information in Cooperative and Byzantine Decentralized Teamingcited by 14
3 more
  1. 2021Deep Adaptive Multi-Intention Inverse Reinforcement Learningcited by 6
  2. 2021Heterogeneous Graph Attention Networks for Learning Diverse Communicationcited by 6
  3. 2017Inverse Reinforce Learning with Nonparametric Behavior Clusteringcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Robots That Take Initiative: A Framework for Building and Evaluating Proactive Robotscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations
  2. 2018Auto-conditioned Recurrent Mixture Density Networks for Learning Generalizable Robot Skills
  3. 2022Lifelong Inverse Reinforcement Learning
  4. 2022Reinforcement learning with Demonstrations from Mismatched Task under Sparse Reward
  5. 2026Continual Policy Consolidation for Lifelong Robot Learning
  6. 2026Self-Distillation Enables Continual Learning
  7. 2026Optimizing Neurorobot Policy under Limited Demonstration Data through Preference Regret
  8. 2024Learning from Demonstration Framework for Multi-Robot Systems Using Interaction Keypoints and Soft Actor-Critic Methods