mlsanityData to 7 Oct 2026

Self-Supervised Policy Adaptation during Deployment

arXiv 2007.04309 ↗First posted 8 Jul 2020cs.LG cs.CV cs.ROcited by 121

Not on the map. Why?

Abstract (arXiv)

In most real world scenarios, a policy trained by reinforcement learning in one environment needs to be deployed in another, potentially quite different environment. However, generalization across different environments is known to be hard. A natural solution would be to keep training after deployment in the new environment, but this cannot be done if the new environment offers no reward signal. Our work explores the use of self-supervision to allow the policy to continue training after deployment without using any rewards. While previous methods explicitly anticipate changes in the new environment, we assume no prior knowledge of those changes yet still obtain significant improvements. Empirical evaluations are performed on diverse simulation environments from DeepMind Control suite and ViZDoom, as well as real robotic manipulation tasks in continuously changing environments, taking observations from an uncalibrated camera. Our method improves generalization in 31 out of 36 environments across various tasks and outperforms domain randomization on a majority of environments.

Key facts. Cited by 121 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 28; 2023: 27; 2024: 12; 2025: 19; 2026: 15.

121 papers.

Sort:

By area:

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  3. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  4. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
  5. 2023Tool Learning with Foundation Modelscited by 302
  6. 2022Test-Time Training with Masked Autoencoderscited by 208
  7. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  8. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
  9. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  10. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
  11. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  12. 2022Test-Time Adaptation via Conjugate Pseudo-labelscited by 98
38 more
  1. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
  2. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
  3. 2025End-to-End Test-Time Training for Long Contextcited by 53
  4. 2021RobustNav: Towards Benchmarking Robustness in Embodied Navigationcited by 52
  5. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  6. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 46
  7. 2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
  8. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
  9. 2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 41
  10. 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 41
  11. 2023Learning Navigational Visual Representations with Semantic Map Supervisioncited by 40
  12. 2021RoMA: Robust Model Adaptation for Offline Model-based Optimizationcited by 39
  13. 2023Finetuning Offline World Models in the Real Worldcited by 39
  14. 2023What Makes Pre-Trained Visual Representations Successful for Robust Manipulation?cited by 38
  15. 2022Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretrainingcited by 37
  16. 2023Test-Time Training on Video Streamscited by 37
  17. 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learningcited by 36
  18. 2021Test-Time Personalization with a Transformer for Human Pose Estimationcited by 33
  19. 2021Unsupervised Visual Attention and Invariance for Reinforcement Learningcited by 30
  20. 2021Towards Distraction-Robust Active Visual Trackingcited by 30
  21. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
  22. 2022Ev-TTA: Test-Time Adaptation for Event-Based Object Recognitioncited by 26
  23. 2023Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Modelscited by 26
  24. 2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
  25. 2022Continuous MDP Homomorphisms and Homomorphic Policy Gradientcited by 24
  26. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  27. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  28. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 23
  29. 2022Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learningcited by 21
  30. 2021Domain Adversarial Reinforcement Learningcited by 19
  31. 2022Temporal Disentanglement of Representations for Improved Generalisation in Reinforcement Learningcited by 19
  32. 2025Test3R: Learning to Reconstruct 3D at Test Timecited by 19
  33. 2022Test-Time Training for Graph Neural Networkscited by 17
  34. 2024Cross-Embodiment Robot Manipulation Skill Transfer using Latent Space Alignmentcited by 17
  35. 2023SNeRL: Semantic-aware Neural Radiance Fields for Reinforcement Learningcited by 16
  36. 2023Curious Replay for Model-based Adaptationcited by 16
  37. 2022Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributionscited by 15
  38. 2022On the Feasibility of Cross-Task Transfer with Model-Based Reinforcement Learningcited by 15

50 most cited of 121 All 121 (JSON)