mlsanityData to 14 Oct 2026

Beyond Domain Randomization: Event-Inspired Perception for Visually Robust Adversarial Imitation from Videos

arXiv 2505.18899 ↗First posted 24 May 2025cs.CV cs.LG cs.ROcited by 1

Not on the map. Why?

Abstract (arXiv)

Imitation from videos often fails when expert demonstrations and learner environments exhibit domain shifts, such as discrepancies in lighting, color, or texture. While visual randomization partially addresses this problem by augmenting training data, it remains computationally intensive and inherently reactive, struggling with unseen scenarios. We propose a different approach: instead of randomizing appearances, we eliminate their influence entirely by rethinking the sensory representation itself. Inspired by biological vision systems that prioritize temporal transients (e.g., retinal ganglion cells) and by recent sensor advancements, we introduce event-inspired perception for visually robust imitation. Our method converts standard RGB videos into a sparse, event-based representation that encodes temporal intensity gradients, discarding static appearance features. This biologically grounded approach disentangles motion dynamics from visual style, enabling robust visual imitation from observations even in the presence of visual mismatches between expert and agent environments. By training policies on event streams, we achieve invariance to appearance-based distractors without requiring computationally expensive and environment-specific data augmentation techniques. Experiments across the DeepMind Control Suite and the Adroit platform for dynamic dexterous manipulation show the efficacy of our method. Our code is publicly available at Eb-LAIfO.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 30 of its 40 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

30 of 40 references.

References · 30

  1. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2016Generative Adversarial Imitation Learningcited by 1,597
  4. 2019Event-based Vision: A Surveycited by 1,016
  5. 2018DeepMind Control Suitecited by 956
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  7. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  8. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 190
  9. 2018Generative Adversarial Imitation from Observationcited by 184
  10. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  11. 2019AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videoscited by 105
  12. 2017Third-Person Imitation Learningcited by 103
18 more
  1. 2022VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learningcited by 52
  2. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreementcited by 46
  3. 2021Cross-domain Imitation from Observationscited by 42
  4. 2021Visual Adversarial Imitation Learning using Variational Modelscited by 42
  5. 2019Learning Belief Representations for Imitation Learning in POMDPscited by 24
  6. 2023Event-based Agile Object Catching with a Quadrupedal Robotcited by 22
  7. 2021Domain-Robust Visual Imitation Learning with Mutual Information Constraintscited by 20
  8. 2023Visual Imitation Learning with Patch Rewardscited by 16
  9. 2024Monocular Event-Based Vision for Obstacle Avoidance with a Quadrotorcited by 14
  10. 2023Deep Visual Odometry with Events and Framescited by 13
  11. 2023SLoMo: A General System for Legged Robot Motion Imitation from Casual Videoscited by 12
  12. 2023Adversarial Imitation Learning from Visual Observations using Latent Informationcited by 11
  13. 2023Domain Adaptive Imitation Learning with Visual Observationcited by 11
  14. 2024Visually Robust Adversarial Imitation Learning from Videos with Contrastive Learningcited by 9
  15. 2024A Recurrent YOLOv8-based framework for Event-Based Object Detectioncited by 9
  16. 2020Domain-Adversarial and Conditional State Space Model for Imitation Learningcited by 7
  17. 2020On the Guaranteed Almost Equivalence between Imitation Learning from Observation and Demonstrationcited by 6
  18. 2022Opportunities and Challenges from Using Animal Videos in Reinforcement Learning for Navigationcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Robust Point Cloud Reinforcement Learning via PCA-Based Canonicalizationcited by 1