mlsanityData to 7 Oct 2026

Invariance Through Latent Alignment

arXiv 2112.08526 ↗First posted 15 Dec 2021cs.LG cs.ROcited by 8

Not on the map. Why?

Abstract (arXiv)

A robot's deployment environment often involves perceptual changes that differ from what it has experienced during training. Standard practices such as data augmentation attempt to bridge this gap by augmenting source images in an effort to extend the support of the training distribution to better cover what the agent might experience at test time. In many cases, however, it is impossible to know test-time distribution-shift a priori, making these schemes infeasible. In this paper, we introduce a general approach, called Invariance Through Latent Alignment (ILA), that improves the test-time performance of a visuomotor control policy in deployment environments with unknown perceptual variations. ILA performs unsupervised adaptation at deployment-time by matching the distribution of latent features on the target domain to the agent's prior experience, without relying on paired data. Although simple, we show that this idea leads to surprising improvements on a variety of challenging adaptation scenarios, including changes in lighting conditions, the content in the scene, and camera poses. We present results on calibrated control benchmarks in simulation -- the distractor control suite -- and a physical robot under a sim-to-real setup.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 15 Dec 2021.

Builds on

27 of 37 references.

References · 27

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2019Invariant Risk Minimizationcited by 1,832
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  5. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  7. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
  8. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  9. 2018Learning Dexterous In-Hand Manipulationcited by 643
  10. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  11. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  12. 2020Reinforcement Learning with Augmented Datacited by 452
15 more
  1. 2019Learning agile and dynamic motor skills for legged robotscited by 402
  2. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  3. 2016FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptationcited by 242
  4. 2019Active Domain Randomizationcited by 161
  5. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 132
  6. 2020Self-Supervised Policy Adaptation during Deploymentcited by 121
  7. 2019Unsupervised Domain Adaptation through Self-Supervisioncited by 116
  8. 2020Invariant Causal Prediction for Block MDPscited by 104
  9. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  10. 2021Learning High-Speed Flight in the Wildcited by 98
  11. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
  12. 2019Learning to Manipulate Deformable Objects without Demonstrationscited by 76
  13. 2020Deep Drone Acrobaticscited by 66
  14. 2018Deep Drone Racing: Learning Agile Flight in Dynamic Environmentscited by 31
  15. 2020Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction timecited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 2; 2026: 1.

8 papers.

  1. 2023What Makes Pre-Trained Visual Representations Successful for Robust Manipulation?cited by 38
  2. 2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 17
  3. 2024Cross-Embodiment Robot Manipulation Skill Transfer using Latent Space Alignmentcited by 17
  4. 2022Learning Representations that Enable Generalization in Assistive Taskscited by 11
  5. 2024R3L: Relative Representations for Reinforcement Learningcited by 2
  6. 2025Mapping representations in Reinforcement Learning via Semantic Alignment for Zero-Shot Stitchingcited by 0
  7. 2025State-Conditional Adversarial Learning: An Off-Policy Visual Domain Transfer Method for End-to-End Imitation Learningcited by 0
  8. 2026Is Success All You Need? Investigating the Impact of Input Perturbations on VLA Behaviour in Tabletop Manipulation Taskscited by 0