mlsanityData to 14 Oct 2026

Unsupervised Representation Learning in Deep Reinforcement Learning: A Review

arXiv 2208.14226 ↗First posted 27 Aug 2022cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

This review addresses the problem of learning abstract representations of the measurement data in the context of Deep Reinforcement Learning (DRL). While the data are often ambiguous, high-dimensional, and complex to interpret, many dynamical systems can be effectively described by a low-dimensional set of state variables. Discovering these state variables from the data is a crucial aspect for (i) improving the data efficiency, robustness, and generalization of DRL methods, (ii) tackling the curse of dimensionality, and (iii) bringing interpretability and insights into black-box DRL. This review provides a comprehensive and complete overview of unsupervised representation learning in DRL by describing the main Deep Learning tools used for learning representations of the world, providing a systematic view of the method and principles, summarizing applications, benchmarks and evaluation strategies, and discussing open challenges and future directions.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 117 of its 170 references that are in the map. First posted on arXiv 27 Aug 2022.

Builds on

117 of 170 references.

References · 117

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  7. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  8. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  9. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  10. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  11. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  12. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
38 more
  1. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  4. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  5. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  6. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
  7. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  8. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  9. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  10. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 763
  11. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  12. 2017Deep Reinforcement Learning that Matterscited by 754
  13. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  14. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  15. 2019Model-Based Reinforcement Learning for Ataricited by 502
  16. 2018Behavioral Cloning from Observationcited by 492
  17. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  18. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  19. 2015Deep Kernel Learningcited by 445
  20. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  21. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  22. 2019Reinforcement Learning in Healthcare: A Surveycited by 310
  23. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  24. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  25. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
  26. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  27. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  28. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  29. 2019Self-Supervised Exploration via Disagreementcited by 265
  30. 2018Visual Reinforcement Learning with Imagined Goalscited by 251
  31. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  32. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  33. 2019Contrastive Learning of Structured World Modelscited by 209
  34. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  35. 2016Learning to Navigate in Complex Environmentscited by 198
  36. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
  37. 2021Reinforcement Learning with Prototypical Representationscited by 179
  38. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174

50 most cited of 117 All 117 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 7; 2026: 3.

18 papers.

Sort:

By area:

  1. 2023Self-Supervised Multimodal Learning: A Surveycited by 58
  2. 2025A Survey of Continual Reinforcement Learningcited by 18
  3. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  4. 2023Trajectory Generation, Control, and Safety with Denoising Diffusion Probabilistic Modelscited by 13
  5. 2025MOBODY: Model Based Off-Dynamics Offline Reinforcement Learningcited by 9
  6. 2024Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policiescited by 7
  7. 2024IsUMap: Manifold Learning and Data Visualization leveraging Vietoris-Rips filtrationscited by 3
  8. 2025HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systemscited by 3
  9. 2023Algebras of actions in an agent's representations of the worldcited by 2
  10. 2024Interpretable and Efficient Data-driven Discovery and Control of Distributed Systemscited by 2
  11. 2024UEVAVD: A Dataset for Developing UAV's Eye View Active Object Detectioncited by 2
  12. 2025Pulling Back the Curtain: Unsupervised Adversarial Detection via Contrastive Auxiliary Networkscited by 2
6 more
  1. 2023CRC-RL: A Novel Visual Feature Representation Architecture for Unsupervised Reinforcement Learningcited by 0
  2. 2025Toward Explainable Offline RL: Analyzing Representations in Intrinsically Motivated Decision Transformerscited by 0
  3. 2025Local Pairwise Distance Matching for Backpropagation-Free Reinforcement Learningcited by 0
  4. 2026Distributional Active Inferencecited by 0
  5. 2026Physics-enhanced reinforcement learning for real-time optimal control of dynamical systemscited by 0
  6. 2026Temporal Logic Guided Universal Task Representations for Reinforcement Learningcited by 0