mlsanityData to 14 Oct 2026

Intrinsic Dynamics-Driven Generalizable Scene Representations for Vision-Oriented Decision-Making Applications

arXiv 2405.19736 ↗First posted 30 May 2024cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

How to improve the ability of scene representation is a key issue in vision-oriented decision-making applications, and current approaches usually learn task-relevant state representations within visual reinforcement learning to address this problem. While prior work typically introduces one-step behavioral similarity metrics with elements (e.g., rewards and actions) to extract task-relevant state information from observations, they often ignore the inherent dynamics relationships among the elements that are essential for learning accurate representations, which further impedes the discrimination of short-term similar task/behavior information in long-term dynamics transitions. To alleviate this problem, we propose an intrinsic dynamics-driven representation learning method with sequence models in visual reinforcement learning, namely DSR. Concretely, DSR optimizes the parameterized encoder by the state-transition dynamics of the underlying system, which prompts the latent encoding information to satisfy the state-transition process and then the state space and the noise space can be distinguished. In the implementation and to further improve the representation ability of DSR on encoding similar tasks, sequential elements' frequency domain and multi-step prediction are adopted for sequentially modeling the inherent dynamics. Finally, experimental results show that DSR has achieved significant performance improvements in the visual Distracting DMControl control tasks, especially with an average of 78.9% over the backbone baseline. Further results indicate that it also achieves the best performances in real-world autonomous driving applications on the CARLA simulator. Moreover, qualitative analysis results validate that our method possesses the superior ability to learn generalizable scene representations on visual tasks. The source code is available at https://github.com/DMU-XMU/DSR.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 30 of its 56 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

30 of 56 references.

References · 30

  1. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  3. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  4. 2018DeepMind Control Suitecited by 956
  5. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  7. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  8. 2022SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentationcited by 171
  9. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  10. 2020Self-Supervised Policy Adaptation during Deploymentcited by 123
  11. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  12. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 100
18 more
  1. 2020S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generationcited by 63
  2. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 57
  3. 2021Towards Robust Bisimulation Metric Learningcited by 49
  4. 2022Understanding Self-Predictive Learning for Reinforcement Learningcited by 44
  5. 2020Masked Contrastive Representation Learning for Reinforcement Learningcited by 32
  6. 2022Investigating the Properties of Neural Network Representations in Reinforcement Learningcited by 24
  7. 2023Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Drivingcited by 17
  8. 2023Instant Domain Augmentation for LiDAR Semantic Segmentationcited by 16
  9. 2022Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributionscited by 15
  10. 2023Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractionscited by 14
  11. 2023Improving Generalization in Visual Reinforcement Learning via Conflict-aware Gradient Agreement Augmentationcited by 14
  12. 2023Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Gamescited by 12
  13. 2023Understanding and Addressing the Pitfalls of Bisimulation-based Representations in Offline Reinforcement Learningcited by 10
  14. 2023Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learningcited by 8
  15. 2023State Sequences Prediction via Fourier Transform for Representation Learningcited by 7
  16. 2022Prediction-aware and Reinforcement Learning based Altruistic Cooperative Drivingcited by 6
  17. 2023Leveraging Reward Consistency for Interpretable Feature Discovery in Reinforcement Learningcited by 3
  18. 2023Sequential Action-Induced Invariant Representation for Reinforcement Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control
  2. 2024The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning
  3. 2025Self-Predictive Dynamics for Generalization of Vision-based Reinforcement Learning
  4. 2024Pre-trained Visual Dynamics Representations for Efficient Policy Learning
  5. 2023Continual Visual Reinforcement Learning with A Life-Long World Model
  6. 2025Pre-trained Visual Representations Generalize Where it Matters in Model-Based Reinforcement Learning
  7. 2024Towards Principled Representation Learning from Videos for Reinforcement Learning
  8. 2022Masked World Models for Visual Control