mlsanityData to 14 Oct 2026

XIRL: Cross-embodiment Inverse Reinforcement Learning

arXiv 2106.03911 ↗First posted 7 Jun 2021cs.RO cs.AI cs.CVcited by 107

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

We investigate the visual cross-embodiment imitation setting, in which agents learn policies from videos of other agents (such as humans) demonstrating the same task, but with stark differences in their embodiments -- shape, actions, end-effector dynamics, etc. In this work, we demonstrate that it is possible to automatically discover and learn vision-based reward functions from cross-embodiment demonstration videos that are robust to these differences. Specifically, we present a self-supervised method for Cross-embodiment Inverse Reinforcement Learning (XIRL) that leverages temporal cycle-consistency constraints to learn deep visual embeddings that capture task progression from offline videos of demonstrations across multiple expert agents, each performing the same task differently due to embodiment differences. Prior to our work, producing rewards from self-supervised embeddings typically required alignment with a reference trajectory, which may be difficult to acquire under stark embodiment differences. We show empirically that if the embeddings are aware of task progress, simply taking the negative distance between the current state and goal state in the learned embedding space is useful as a reward for training policies with reinforcement learning. We find our learned reward function not only works for embodiments seen during training, but also generalizes to entirely new embodiments. Additionally, when transferring real-world human demonstrations to a simulated robot, we find that XIRL is more sample efficient than current best methods. Qualitative results, code, and datasets are available at https://x-irl.github.io

Key facts. Cited by 107 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 40 of its 58 references that are in the map. First posted on arXiv 7 Jun 2021.

Builds on

40 of 58 references.

Closest ancestors · 4

Same direction, newest first

  1. 2020State-Only Imitation Learning for Dexterous Manipulation7 Apr
  2. 2019Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller21 Nov
  3. 2019Self-Supervised Correspondence in Visuomotor Policy Learning16 Sep
  4. 2018Zero-Shot Visual Imitation23 Apr

Other references · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  7. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  8. 2016Generative Adversarial Imitation Learningcited by 1,597
  9. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  10. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 763
  11. 2017Deep Reinforcement Learning that Matterscited by 754
  12. 2018Albumentations: fast and flexible image augmentationscited by 560
24 more
  1. 2018Behavioral Cloning from Observationcited by 492
  2. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  3. 2018An Algorithmic Perspective on Imitation Learningcited by 459
  4. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 289
  5. 2018Generative Adversarial Imitation from Observationcited by 184
  6. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  7. 2019Temporal Cycle-Consistency Learningcited by 154
  8. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 122
  9. 2019AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videoscited by 105
  10. 2017Third-Person Imitation Learningcited by 103
  11. 2018Imitating Latent Policies from Observationcited by 101
  12. 2018Playing hard exploration games by watching YouTubecited by 99
  13. 2017Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learningcited by 85
  14. 2020Reinforcement Learning with Videos: Combining Offline Observations with Interactioncited by 82
  15. 2019Provably Efficient Imitation Learning from Observation Alonecited by 78
  16. 2019State Alignment-based Imitation Learningcited by 73
  17. 2016Unsupervised Perceptual Rewards for Imitation Learningcited by 65
  18. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreementcited by 46
  19. 2018A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learningcited by 36
  20. 2018Learning Actionable Representations from Visual Observationscited by 35
  21. 2020The MAGICAL Benchmark for Robust Imitationcited by 32
  22. 2020Hierarchically Decoupled Imitation for Morphological Transfercited by 31
  23. 2019Adversarial Skill Networks: Unsupervised Robot Skill Learning from Videocited by 17
  24. 2020ADAIL: Adaptive Adversarial Imitation Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 17; 2023: 24; 2024: 28; 2025: 23; 2026: 11.

107 papers.

Sort:

By area:

  1. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  2. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  3. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  4. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  5. 2022Visual Language Maps for Robot Navigationcited by 365
  6. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 223
  7. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 164
  8. 2022Human-to-Robot Imitation in the Wildcited by 161
  9. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 148
  10. 2023Structured World Models from Human Videoscited by 147
  11. 2024Flow as the Cross-Domain Manipulation Interfacecited by 132
  12. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
38 more
  1. 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 90
  2. 2022Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtubecited by 82
  3. 2025Humanoid Policy ~ Human Policycited by 78
  4. 2023XSkill: Cross Embodiment Skill Discoverycited by 75
  5. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 68
  6. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 62
  7. 2025Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learningcited by 59
  8. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusioncited by 57
  9. 2021Human Hands as Probes for Interactive Object Understandingcited by 52
  10. 2024Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformerscited by 52
  11. 2021Cross-Domain Imitation Learning via Optimal Transportcited by 49
  12. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 46
  13. 2022Reinforcement Learning with Neural Radiance Fieldscited by 43
  14. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 43
  15. 2023GROOT: Learning to Follow Instructions by Watching Gameplay Videoscited by 42
  16. 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learningcited by 41
  17. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  18. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 38
  19. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  20. 2024ViViDex: Learning Vision-based Dexterous Manipulation from Human Videoscited by 35
  21. 2025Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstrationcited by 34
  22. 2024Rank2Reward: Learning Shaped Reward Functions from Passive Videocited by 32
  23. 2023Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitationcited by 31
  24. 2024CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulationcited by 28
  25. 2025X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Realcited by 28
  26. 2023AR2-D2:Training a Robot Without a Robotcited by 27
  27. 2024Cross-Embodiment Dexterous Grasping with Reinforcement Learningcited by 27
  28. 2024Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulationcited by 23
  29. 2023For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equalcited by 22
  30. 2022Visuomotor Control in Multi-Object Scenes Using Object-Aware Representationscited by 21
  31. 2023Look Ma, No Hands! Agent-Environment Factorization of Egocentric Videoscited by 21
  32. 2022Improving Policy Optimization with Generalist-Specialist Learningcited by 20
  33. 2024Video-Language Critic: Transferable Reward Functions for Language-Conditioned Roboticscited by 19
  34. 2024Cross-Embodiment Robot Manipulation Skill Transfer using Latent Space Alignmentcited by 18
  35. 2022Learning Reward Functions for Robotic Manipulation by Observing Humanscited by 17
  36. 2024A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agentscited by 16
  37. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 16
  38. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddingscited by 14

50 most cited of 107 All 107 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Cross-Embodiment Transfer via Behavior-Aligned Representations
  2. 2024Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment
  3. 2021Manipulator-Independent Representations for Visual Imitation
  4. 2026CLAP: Cross-Embodiment Video World Models are Zero-Shot Physical Simulators
  5. 2025Consistent Zero-Shot Imitation with Contrastive Goal Inference
  6. 2024Meta-Controller: Few-Shot Imitation of Unseen Embodiments and Tasks in Continuous Control
  7. 2025NIL: No-data Imitation Learning by Leveraging Pre-trained Video Diffusion Models
  8. 2026Cross-Embodiment Offline Reinforcement Learning for Heterogeneous Robot Datasets