mlsanityData to 14 Oct 2026

On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learning

arXiv 2306.05637 ↗First posted 9 Jun 2023cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Recently, unsupervised representation learning (URL) has improved the sample efficiency of Reinforcement Learning (RL) by pretraining a model from a large unlabeled dataset. The underlying principle of these methods is to learn temporally predictive representations by predicting future states in the latent space. However, an important challenge of this approach is the representational collapse, where the subspace of the latent representations collapses into a low-dimensional manifold. To address this issue, we propose a novel URL framework that causally predicts future states while increasing the dimension of the latent manifold by decorrelating the features in the latent space. Through extensive empirical studies, we demonstrate that our framework effectively learns predictive representations without collapse, which significantly improves the sample efficiency of state-of-the-art URL methods on the Atari 100k benchmark. The code is available at https://github.com/dojeon-ai/SimTPR.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 53 of its 72 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

53 of 72 references.

References · 53

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2013Auto-Encoding Variational Bayescited by 4,053
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  10. 2020Supervised Contrastive Learningcited by 2,829
  11. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  12. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
38 more
  1. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  4. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  5. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  7. 2018DeepMind Control Suitecited by 956
  8. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  9. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  10. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  11. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  12. 2019Model-Based Reinforcement Learning for Ataricited by 502
  13. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  14. 2020Understanding the Behaviour of Contrastive Losscited by 398
  15. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 378
  16. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  17. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  18. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  19. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  20. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  21. 2023Understanding plasticity in neural networkscited by 174
  22. 2021On Feature Decorrelation in Self-Supervised Learningcited by 160
  23. 2022RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rankcited by 153
  24. 2019Unsupervised State Representation Learning in Ataricited by 144
  25. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 125
  26. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  27. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
  28. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  29. 2015Optimal whitening and decorrelationcited by 97
  30. 2020BYOL works even without batch statisticscited by 95
  31. 2015Reducing Overfitting in Deep Networks by Decorrelating Representationscited by 94
  32. 2018Decorrelated Batch Normalizationcited by 83
  33. 2020Transient Non-Stationarity and Generalisation in Deep Reinforcement Learningcited by 75
  34. 2017Deep Reinforcement Learning for Sepsis Treatmentcited by 71
  35. 2018Probabilistic Recurrent State-Space Modelscited by 64
  36. 2016Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Modelcited by 54
  37. 2019Learning When-to-Treat Policiescited by 49
  38. 2018Learning and Querying Fast Generative Models for Reinforcement Learningcited by 48

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 3; 2026: 1.

8 papers.

  1. 2023PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learningcited by 42
  2. 2023Learning to Discover Skills through Guidancecited by 8
  3. 2024Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learningcited by 4
  4. 2025TransDiffuser: Diverse Trajectory Generation with Decorrelated Multi-modal Representation for End-to-end Autonomous Drivingcited by 4
  5. 2025Flatness After All?cited by 1
  6. 2026Enhancing Time Series Classification with Diversity-Driven Neural Network Ensemblescited by 1
  7. 2024Uncovering RL Integration in SSL Loss: Objective-Specific Implications for Data-Efficient RLcited by 0
  8. 2025Learning Distinguishable Representations in Deep Q-Networks for Linear Transfercited by 0