mlsanityData to 7 Oct 2026

Uncovering RL Integration in SSL Loss: Objective-Specific Implications for Data-Efficient RL

arXiv 2410.17428 ↗First posted 22 Oct 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this study, we investigate the effect of SSL objective modifications within the SPR framework, focusing on specific adjustments such as terminal state masking and prioritized replay weighting, which were not explicitly addressed in the original design. While these modifications are specific to RL, they are not universally applicable across all RL algorithms. Therefore, we aim to assess their impact on performance and explore other SSL objectives that do not accommodate these adjustments like Barlow Twins and VICReg. We evaluate six SPR variants on the Atari 100k benchmark, including versions both with and without these modifications. Additionally, we test the performance of these objectives on the DeepMind Control Suite, where such modifications are absent. Our findings reveal that incorporating specific SSL modifications within SPR significantly enhances performance, and this influence extends to subsequent frameworks like SR-SPR and BBF, highlighting the critical importance of SSL objective selection and related adaptations in achieving data efficiency in self-predictive reinforcement learning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 46 of its 52 references that are in the map. First posted on arXiv 22 Oct 2024.

Builds on

46 of 52 references.

References · 46

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  7. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  8. 2023Mastering Diverse Domains through World Modelscited by 1,193
  9. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  10. 2018DeepMind Control Suitecited by 935
  11. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  12. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
34 more
  1. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  2. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  3. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  4. 2019Model-Based Reinforcement Learning for Ataricited by 495
  5. 2022Transformers are Sample-Efficient World Modelscited by 317
  6. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  7. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  8. 2023A Cookbook of Self-Supervised Learningcited by 248
  9. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  10. 2021Mastering Atari Games with Limited Datacited by 235
  11. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
  12. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  13. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 137
  14. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  15. 2019When to use parametric models in reinforcement learning?cited by 118
  16. 2021APS: Active Pretraining with Successor Featurescited by 109
  17. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  18. 2021PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learningcited by 41
  19. 2022Self-Supervised Learning with an Information Maximization Criterioncited by 38
  20. 2022Stabilizing Off-Policy Deep Reinforcement Learning from Pixelscited by 32
  21. 2022Hyperbolic Deep Reinforcement Learningcited by 25
  22. 2022Non-Contrastive Learning Meets Language-Image Pre-Trainingcited by 25
  23. 2021Learning Representations for Pixel-based Control: What Matters and Why?cited by 24
  24. 2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
  25. 2023Matrix Information Theory for Self-Supervised Learningcited by 21
  26. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learningcited by 21
  27. 2022Reducing Variance in Temporal-Difference Value Estimation via Ensemble of Deep Networkscited by 13
  28. 2023Modulate Your Spectrum in Self-Supervised Learningcited by 9
  29. 2023On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learningcited by 8
  30. 2024Learning from Memory: Non-Parametric Memory Augmented Self-Supervised Learning of Visual Featurescited by 6
  31. 2022Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learningcited by 5
  32. 2022Accelerating Representation Learning with View-Consistent Dynamics in Data-Efficient Reinforcement Learningcited by 3
  33. 2023BarlowRL: Barlow Twins for Data-Efficient Reinforcement Learningcited by 2
  34. 2021Is Policy Learning Overrated?: Width-Based Planning and Active Learning for Ataricited by 1

Built on by

Not cited yet.