mlsanityData to 7 Oct 2026

Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatch

arXiv 2104.07810 ↗First posted 15 Apr 2021cs.LG cs.ROcited by 5

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Learning from demonstrations in the wild (e.g. YouTube videos) is a tantalizing goal in imitation learning. However, for this goal to be achieved, imitation learning algorithms must deal with the fact that the demonstrators and learners may have bodies that differ from one another. This condition -- "embodiment mismatch" -- is ignored by many recent imitation learning algorithms. Our proposed imitation learning technique, SILEM (Skeletal feature compensation for Imitation Learning with Embodiment Mismatch), addresses a particular type of embodiment mismatch by introducing a learned affine transform to compensate for differences in the skeletal features obtained from the learner and expert. We create toy domains based on PyBullet's HalfCheetah and Ant to assess SILEM's benefits for this type of embodiment mismatch. We also provide qualitative and quantitative results on more realistic problems -- teaching simulated humanoid agents, including Atlas from Boston Dynamics, to walk by observing human demonstrations.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 22 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

17 of 22 references.

References · 17

  1. 2017Deep reinforcement learning from human preferencescited by 3,846
  2. 2014Conditional Generative Adversarial Netscited by 2,152
  3. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2018DeepMind Control Suitecited by 935
  6. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  7. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  8. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 229
  9. 2018Generative Adversarial Imitation from Observationcited by 183
  10. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  11. 2017Third-Person Imitation Learningcited by 103
  12. 2017Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learningcited by 84
5 more
  1. 2017Learning human behaviors from motion capture by adversarial imitationcited by 75
  2. 2019State Alignment-based Imitation Learningcited by 73
  3. 2018Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translationcited by 50
  4. 2020State-only Imitation with Transition Dynamics Mismatchcited by 37
  5. 2021Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platformscited by 22

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2; 2023: 0; 2024: 1.

5 papers.

  1. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddingscited by 14
  2. 2021Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observationscited by 4
  3. 2022Co-Imitation: Learning Design and Behaviour by Imitationcited by 3
  4. 2021RAIL: A modular framework for Reinforcement-learning-based Adversarial Imitation Learningcited by 2
  5. 2024Generalized Regression with Conditional GANscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Metric-Based Imitation Learning Between Two Dissimilar Anthropomorphic Robotic Arms
  2. 2024Representation Alignment from Human Feedback for Cross-Embodiment Reward Learning from Mixed-Quality Demonstrations
  3. 2022Eliciting Compatible Demonstrations for Multi-Human Imitation Learning
  4. 2025NIL: No-data Imitation Learning by Leveraging Pre-trained Video Diffusion Models
  5. 2019Domain Adaptive Imitation Learning
  6. 2025Imitation Learning from a Single Temporally Misaligned Video
  7. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations
  8. 2023XSkill: Cross Embodiment Skill Discovery