mlsanityData to 7 Oct 2026

When a Robot is More Capable than a Human: Learning from Constrained Demonstrators

arXiv 2510.09096 ↗First posted 10 Oct 2025cs.RO cs.AI cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Robot learning from demonstration

Abstract (arXiv)

Learning from demonstrations enables experts to teach robots complex tasks using interfaces such as kinesthetic teaching, joystick control, and sim-to-real transfer. However, these interfaces often constrain the expert's ability to demonstrate optimal behavior due to indirect control, setup restrictions, and hardware safety. For example, a joystick can move a robotic arm only in a 2D plane, even though the robot operates in a higher-dimensional space. As a result, the demonstrations collected by constrained experts lead to suboptimal performance of the learned policies. This raises a key question: Can a robot learn a better policy than the one demonstrated by a constrained expert? We address this by allowing the agent to go beyond direct imitation of expert actions and explore shorter and more efficient trajectories. We use the demonstrations to infer a state-only reward signal that measures task progress, and self-label reward for unknown states using temporal interpolation. Our approach outperforms common imitation learning in both sample efficiency and task completion time. On a real WidowX robotic arm, it completes the task in 12 seconds, 10x faster than behavioral cloning, as shown in real-robot videos on https://sites.google.com/view/constrainedexpert .

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data. Builds on 32 of its 47 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

32 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regression17 Oct

Other references · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2018Behavioral Cloning from Observationcited by 482
  7. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  8. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  9. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
  10. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  11. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  12. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
19 more
  1. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  2. 2018Generative Adversarial Imitation from Observationcited by 183
  3. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  4. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 159
  5. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 101
  6. 2018Reinforcement Learning from Imperfect Demonstrationscited by 74
  7. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  8. 2019State Alignment-based Imitation Learningcited by 73
  9. 2020Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferencescited by 71
  10. 2025ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrationscited by 52
  11. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreementcited by 46
  12. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimalitycited by 44
  13. 2021Cross-domain Imitation from Observationscited by 42
  14. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
  15. 2025CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrationscited by 39
  16. 2023Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matchingcited by 24
  17. 2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representationscited by 17
  18. 2025SAIL: Faster-than-Demonstration Execution of Imitation Learning Policiescited by 16
  19. 2024Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functionscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Learning from Imperfect Demonstrations from Agents with Varying Dynamics
  2. 2022Unified Learning from Demonstrations, Corrections, and Preferences during Physical Human-Robot Interaction
  3. 2021Learning Online from Corrective Feedback: A Meta-Algorithm for Robotics
  4. 2021Learning from Successful and Failed Demonstrations via Optimization
  5. 2018Constraint Estimation and Derivative-Free Recovery for Robot Learning from Demonstrations
  6. 2021Ergodic imitation: Learning from what to do and what not to do
  7. 2022Imitation Learning by Estimating Expertise of Demonstrators
  8. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets