mlsanityData to 7 Oct 2026

Towards Sample-efficient Apprenticeship Learning from Suboptimal Demonstration

arXiv 2110.04347 ↗First posted 8 Oct 2021cs.RO cs.LGcited by 1

On the map: Offline RL and imitation learning from data → Robot learning from demonstration

Abstract (arXiv)

Learning from Demonstration (LfD) seeks to democratize robotics by enabling non-roboticist end-users to teach robots to perform novel tasks by providing demonstrations. However, as demonstrators are typically non-experts, modern LfD techniques are unable to produce policies much better than the suboptimal demonstration. A previously-proposed framework, SSRR, has shown success in learning from suboptimal demonstration but relies on noise-injected trajectories to infer an idealized reward function. A random approach such as noise-injection to generate trajectories has two key drawbacks: 1) Performance degradation could be random depending on whether the noise is applied to vital states and 2) Noise-injection generated trajectories may have limited suboptimality and therefore will not accurately represent the whole scope of suboptimality. We present Systematic Self-Supervised Reward Regression, S3RR, to investigate systematic alternatives for trajectory degradation. We carry out empirical evaluations and find S3RR can learn comparable or better reward correlation with ground-truth against a state-of-the-art learning from suboptimal demonstration framework.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data. Builds on 9 of its 29 references that are in the map. First posted on arXiv 8 Oct 2021.

Builds on

9 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regression17 Oct

Other references · 8

  1. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  2. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  3. 2017Game-Theoretic Modeling of Human Adaptation in Human-Robot Collaborationcited by 21
  4. 2020Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillationcited by 19
  5. 2018Towards Robotic Feeding: Role of Haptics in Fork-based Food Manipulationcited by 16
  6. 2019Interpretable and Personalized Apprenticeship Scheduling: Learning Interpretable Scheduling Policies from Heterogeneous User Demonstrationscited by 14
  7. 2020Coordinated Control of UAVs for Human-Centered Active Sensing of Wildfirescited by 8
  8. 2019Measuring the Data Efficiency of Deep Learning Methodscited by 5

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Strategy Discovery and Mixture in Lifelong Learning from Heterogeneous Demonstrationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2022Quantifying Demonstration Quality for Robot Learning and Generalization
  2. 2021Learning from Successful and Failed Demonstrations via Optimization
  3. 2024Consistency Matters: Defining Demonstration Data Quality Metrics in Robot Learning from Demonstration
  4. 2024Stochastic Trajectory Optimization for Robotic Skill Acquisition From a Suboptimal Demonstration
  5. 2025Demonstration Sidetracks: Categorizing Systematic Non-Optimality in Human Demonstrations
  6. 2024Reward Learning from Suboptimal Demonstrations with Applications in Surgical Electrocautery
  7. 2022Autonomous Assessment of Demonstration Sufficiency via Bayesian Inverse Reinforcement Learning
  8. 2023Learning to Discern: Imitating Heterogeneous Human Demonstrations with Preference and Representation Learning