mlsanityData to 14 Oct 2026

Demonstration Sidetracks: Categorizing Systematic Non-Optimality in Human Demonstrations

arXiv 2506.11262 ↗First posted 12 Jun 2025cs.RO cs.LGcited by 2

On the map: Offline RL and imitation learning from data → Robot learning from demonstration

Abstract (arXiv)

Learning from Demonstration (LfD) is a popular approach for robots to acquire new skills, but most LfD methods suffer from imperfections in human demonstrations. Prior work typically treats these suboptimalities as random noise. In this paper we study non-optimal behaviors in non-expert demonstrations and show that they are systematic, forming what we call demonstration sidetracks. Using a public space study with 40 participants performing a long-horizon robot task, we recreated the setup in simulation and annotated all demonstrations. We identify four types of sidetracks (Exploration, Mistake, Alignment, Pause) and one control pattern (one-dimension control). Sidetracks appear frequently across participants, and their temporal and spatial distribution is tied to task context. We also find that users' control patterns depend on the control interface. These insights point to the need for better models of suboptimal demonstrations to improve LfD algorithms and bridge the gap between lab training and real-world deployment. All demonstrations, infrastructure, and annotations are available at https://github.com/AABL-Lab/Human-Demonstration-Sidetracks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 37 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

23 of 37 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regression17 Oct

Other references · 22

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  2. 2016Generative Adversarial Imitation Learningcited by 1,597
  3. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  4. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  5. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  6. 2021Implicit Behavioral Cloningcited by 452
  7. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 260
  8. 2017DART: Noise Injection for Robust Imitation Learningcited by 191
  9. 2019Imitation Learning from Imperfect Demonstrationcited by 114
  10. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 93
  11. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrationscited by 89
  12. 2022Imitation Learning by Estimating Expertise of Demonstratorscited by 50
10 more
  1. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimalitycited by 44
  2. 2023Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matchingcited by 24
  3. 2023Learning to Discern: Imitating Heterogeneous Human Demonstrations with Preference and Representation Learningcited by 15
  4. 2024Inverse Reinforcement Learning by Estimating Expertise of Demonstratorscited by 11
  5. 2022Robust Imitation Learning from Corrupted Demonstrationscited by 10
  6. 2023From "Thumbs Up" to "10 out of 10": Reconsidering Scalar Feedback in Interactive Reinforcement Learningcited by 6
  7. 2023Identifying Expert Behavior in Offline Training Datasets Improves Behavioral Cloning of Robotic Manipulation Policiescited by 5
  8. 2023Good Better Best: Self-Motivated Imitation Learning for noisy Demonstrationscited by 4
  9. 2023Aligning Human Intent from Imperfect Demonstrations with Confidence-based Inverse soft-Q Learningcited by 2
  10. 2024How Much Progress Did I Make? An Unexplored Human Feedback Signal for Teaching Robotscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025ACG: Action Coherence Guidance for Flow-based Vision-Language-Action modelscited by 1
  2. 2026PHIRL: Aligning Learned Rewards with Task Progress for Inverse Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Learning from Successful and Failed Demonstrations via Optimization
  2. 2021Ergodic imitation: Learning from what to do and what not to do
  3. 2025When a Robot is More Capable than a Human: Learning from Constrained Demonstrators
  4. 2019Benchmark for Skill Learning from Demonstration: Impact of User Experience, Task Complexity, and Start Configuration on Performance
  5. 2021Towards Sample-efficient Apprenticeship Learning from Suboptimal Demonstration
  6. 2022Quantifying Demonstration Quality for Robot Learning and Generalization
  7. 2021Learning from Imperfect Demonstrations from Agents with Varying Dynamics
  8. 2025Action Chunking and Exploratory Data Collection Yield Exponential Improvements in Behavior Cloning for Continuous Control