mlsanityData to 14 Oct 2026

SimSR: Simple Distance-based State Representation for Deep Reinforcement Learning

arXiv 2112.15303 ↗First posted 31 Dec 2021cs.LG cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

This work explores how to learn robust and generalizable state representation from image-based observations with deep reinforcement learning methods. Addressing the computational complexity, stringent assumptions and representation collapse challenges in existing work of bisimulation metric, we devise Simple State Representation (SimSR) operator. SimSR enables us to design a stochastic approximation method that can practically learn the mapping functions (encoders) from observations to latent representation space. In addition to the theoretical analysis and comparison with the existing work, we experimented and compared our work with recent state-of-the-art solutions in visual MuJoCo tasks. The results shows that our model generally achieves better performance and has better robustness and good generalization.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 28 of its 43 references that are in the map. First posted on arXiv 31 Dec 2021.

Builds on

28 of 43 references.

References · 28

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  3. 2017The Kinetics Human Action Video Datasetcited by 1,909
  4. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  5. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  8. 2018DeepMind Control Suitecited by 956
  9. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  10. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  11. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  12. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
16 more
  1. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  2. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  3. 2017NormFace: L2 Hypersphere Embedding for Face Verificationcited by 237
  4. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  5. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  6. 2021Reinforcement Learning with Prototypical Representationscited by 179
  7. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  8. 2019Unsupervised State Representation Learning in Ataricited by 144
  9. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  10. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  11. 2017Unsupervised Learning by Predicting Noisecited by 98
  12. 2016Loss is its own Reward: Self-Supervision for Reinforcement Learningcited by 72
  13. 2020Predictive Information Accelerates Learning in RLcited by 56
  14. 2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 44
  15. 2021MICo: Improved representations via sampling-based state similarity for Markov decision processescited by 43
  16. 2021DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneckcited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 2; 2025: 5; 2026: 1.

14 papers.

Sort:

By area:

  1. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 41
  2. 2024Learning Latent Dynamic Robust Representations for World Modelscited by 21
  3. 2023Hierarchical State Abstraction Based on Structural Information Principlescited by 14
  4. 2022Behavior Prior Representation learning for Offline Reinforcement Learningcited by 12
  5. 2023Understanding and Addressing the Pitfalls of Bisimulation-based Representations in Offline Reinforcement Learningcited by 10
  6. 2025Reward-Aware Proto-Representations in Reinforcement Learningcited by 4
  7. 2023Sequential Action-Induced Invariant Representation for Reinforcement Learningcited by 3
  8. 2024State Chrono Representation for Enhancing Generalization in Reinforcement Learningcited by 3
  9. 2025Revisiting Bisimulation Metric for Robust Representations in Reinforcement Learningcited by 3
  10. 2025Learning Fused State Representations for Control from Multi-View Observationscited by 1
  11. 2023Wasserstein Auto-encoded MDPs: Formal Verification of Efficiently Distilled RL Policies with Many-sided Guaranteescited by 0
  12. 2023Towards Control-Centric Representations in Reinforcement Learning from Imagescited by 0
2 more
  1. 2025Autonomous Learning From Success and Failure: Goal-Conditioned Supervised Learning with Negative Feedbackcited by 0
  2. 2026PAMD: Structured Adaptive Distances for Bisimulation Representations in Visual Reinforcement Learningcited by 0