mlsanityData to 7 Oct 2026

Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation

arXiv 2102.05714 ↗First posted 10 Feb 2021cs.LG cs.AIcited by 33

Not on the map. Why?

Abstract (arXiv)

Despite the recent success of deep reinforcement learning (RL), domain adaptation remains an open problem. Although the generalization ability of RL agents is critical for the real-world applicability of Deep RL, zero-shot policy transfer is still a challenging problem since even minor visual changes could make the trained agent completely fail in the new task. To address this issue, we propose a two-stage RL agent that first learns a latent unified state representation (LUSR) which is consistent across multiple domains in the first stage, and then do RL training in one source domain based on LUSR in the second stage. The cross-domain consistency of LUSR allows the policy acquired from the source domain to generalize to other target domains without extra training. We first demonstrate our approach in variants of CarRacing games with customized manipulations, and then verify it in CARLA, an autonomous driving simulator with more complex and realistic visual observations. Our results show that this approach can achieve state-of-the-art domain adaptation performance in related RL tasks and outperforms prior approaches based on latent-representation based RL and image-to-image translation.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 16 of its 23 references that are in the map. First posted on arXiv 10 Feb 2021.

Builds on

16 of 23 references.

References · 16

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  6. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  7. 2018Learning Dexterous In-Hand Manipulationcited by 643
  8. 2017Unsupervised Image-to-Image Translation Networkscited by 602
  9. 2020Reinforcement Learning with Augmented Datacited by 452
  10. 2017DARLA: Improving Zero-Shot Transfer in Reinforcement Learningcited by 147
  11. 2017Visualizing and Understanding Atari Agentscited by 141
  12. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 96
4 more
  1. 2017Virtual to Real Reinforcement Learning for Autonomous Drivingcited by 79
  2. 2018Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translationcited by 50
  3. 2018Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoderscited by 49
  4. 2015Adapting Deep Visuomotor Representations with Weak Pairwise Constraintscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 8; 2024: 7; 2025: 8; 2026: 4.

33 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2024Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitationcited by 15
  3. 2023MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learningcited by 9
  4. 2024Beyond the Edge: An Advanced Exploration of Reinforcement Learning for Mobile Edge Computing, its Applications, and Future Research Trajectoriescited by 9
  5. 2025Zero-Shot Adaptation of Behavioral Foundation Models to Unseen Dynamicscited by 8
  6. 2024Model-Based Transfer Learning for Contextual Reinforcement Learningcited by 7
  7. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulatorcited by 7
  8. 2022No More Pesky Hyperparameters: Offline Hyperparameter Tuning for RLcited by 6
  9. 2024Efficient Policy Adaptation with Contrastive Prompt Ensemble for Embodied Agentscited by 5
  10. 2021Domain Generalization for Vision-based Driving Trajectory Generationcited by 4
  11. 2022Opportunities and Challenges from Using Animal Videos in Reinforcement Learning for Navigationcited by 4
  12. 2023Online Prototype Alignment for Few-shot Policy Transfercited by 4
21 more
  1. 2023A Framework for Few-Shot Policy Transfer through Observation Mapping and Behavior Cloningcited by 4
  2. 2023Automaton Distillation: Neuro-Symbolic Transfer Learning for Deep Reinforcement Learningcited by 4
  3. 2024Offline Reinforcement Learning with Domain-Unlabeled Datacited by 3
  4. 2023P3O: Transferring Visual Representations for Reinforcement Learning via Promptingcited by 1
  5. 2025Curriculum-Adapted Robust Reinforcement Learning for UAV Deconfliction in Adversarial Environmentscited by 1
  6. 2022Malleable Agents for Re-Configurable Robotic Manipulatorscited by 0
  7. 2022Model Generation with Provable Coverability for Offline Reinforcement Learningcited by 0
  8. 2022Unified State Representation Learning under Data Augmentationcited by 0
  9. 2023Zero-Shot Transfer in Imitation Learningcited by 0
  10. 2023Imitation learning for sim-to-real transfer of robotic cutting policies based on residual Gaussian process disturbance force modelcited by 0
  11. 2024Automatic re-calibration of quantum devices by reinforcement learningcited by 0
  12. 2024Prompt-based Visual Alignment for Zero-shot Policy Transfercited by 0
  13. 2025Planning Under Observation Mismatch for Traffic Signal Control via Adaptive Modular World Modelscited by 0
  14. 2025CooT: Learning to Coordinate In-Context with Coordination Transformerscited by 0
  15. 2025Reinforcement Learning Driven Generalizable Feature Representation for Cross-User Activity Recognitioncited by 0
  16. 2025DeGuV: Depth-Guided Visual Reinforcement Learning for Generalization and Interpretability in Manipulationcited by 0
  17. 2025Vehicle Dynamics Embedded World Models for Autonomous Drivingcited by 0
  18. 2026Sim-to-Real Transfer via a Style-Identified Cycle Consistent Generative Adversarial Network: Zero-Shot Deployment on Robotic Manipulators through Visual Domain Adaptationcited by 0
  19. 2026The Constant Eye: Benchmarking and Bridging Appearance Robustness in Autonomous Drivingcited by 0
  20. 2026Bridging Dynamics Gaps via Diffusion Schr"odinger Bridge for Cross-Domain Reinforcement Learningcited by 0
  21. 2026Intrinsic Vicarious Conditioning for Deep Reinforcement Learningcited by 0