mlsanityData to 14 Oct 2026

Reinforcement Learning from Cross-domain Videos with Video Prediction Model

arXiv 2606.03201 ↗First posted 2 Jun 2026cs.CV cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Reinforcement learning from expert videos across visually distinct domains is challenging due to the absence of reward signals and the presence of domain gaps. We introduce XIPER (Cross-domain Video Prediction Reward), a reward model for learning from expert videos collected in a visually different domain, where the agent's appearance differs due to factors such as color, morphology, or the sim-to-real gap. More specifically, XIPER trains a cross-domain video prediction model that maps agent observations into the expert domain and uses the prediction likelihood as a reward signal. Experiments on the DMC Color Suite (8 tasks) and DMC Body Suite (3 tasks) show that XIPER consistently outperforms baselines despite domain gaps such as differences in agent color and morphology. We further analyze XIPER on a sim-to-real transfer dataset, demonstrating that it produces meaningful reward signals for real-robot observations given only simulated expert videos. Code, pretrained models, datasets and video demonstrations can be found on our project webpage: https://sites.google.com/view/xiper

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 39 of its 51 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

39 of 51 references.

References · 39

  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  2. 2023Mastering Diverse Domains through World Modelscited by 1,253
  3. 2018Exploration by Random Network Distillationcited by 993
  4. 2018Which Training Methods for GANs do actually Converge?cited by 679
  5. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  6. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  7. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  8. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  9. 2022MAGVIT: Masked Generative Video Transformercited by 346
  10. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  11. 2018An Introduction to Deep Reinforcement Learningcited by 206
  12. 2018Generative Adversarial Imitation from Observationcited by 184
27 more
  1. 2025Flow Q-Learningcited by 184
  2. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  3. 2022Neural Optimal Transportcited by 126
  4. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
  5. 2019AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videoscited by 105
  6. 2017Third-Person Imitation Learningcited by 103
  7. 2023Unpaired Image-to-Image Translation via Neural Schr"odinger Bridgecited by 96
  8. 2020RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Realcited by 91
  9. 2020Reinforcement Learning with Videos: Combining Offline Observations with Interactioncited by 82
  10. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 78
  11. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 76
  12. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learningcited by 74
  13. 2020RetinaGAN: An Object-aware Approach to Sim-to-Real Transfercited by 70
  14. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusioncited by 57
  15. 2021Domain Invariant Representation Learning with Domain Density Transformationscited by 53
  16. 2018Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translationcited by 50
  17. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  18. 2021Cross-domain Imitation from Observationscited by 42
  19. 2024Video Diffusion Models: A Surveycited by 30
  20. 2021Domain-Robust Visual Imitation Learning with Mutual Information Constraintscited by 20
  21. 2021Domain Adversarial Reinforcement Learningcited by 19
  22. 2024ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Modelscited by 19
  23. 2021Imitation by Predicting Observationscited by 13
  24. 2024D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learningcited by 12
  25. 2023Domain Adaptive Imitation Learning with Visual Observationcited by 11
  26. 2024Visually Robust Adversarial Imitation Learning from Videos with Contrastive Learningcited by 9
  27. 2022Sim-to-Real Transfer of Robotic Assembly with Visual Inputs Using CycleGAN and Force Controlcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021XIRL: Cross-embodiment Inverse Reinforcement Learning
  2. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddings
  3. 2025X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Real
  4. 2025Goal-Driven Reward by Video Diffusion Models for Reinforcement Learning
  5. 2022Graph Inverse Reinforcement Learning from Diverse Videos
  6. 2025Reinforcement Learning with Inverse Rewards for World Model Post-training
  7. 2024Video-Language Critic: Transferable Reward Functions for Language-Conditioned Robotics
  8. 2026Semi-Supervised Cross-Domain Imitation Learning