mlsanityData to 14 Oct 2026

R2R2: Robust Representation for Intensive Experience Reuse via Redundancy Reduction in Self-Predictive Learning

arXiv 2605.14026 ↗First posted 13 May 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

For reinforcement learning in data-scarce domains like real-world robotics, intensive data reuse enhances efficiency but induces overfitting. While prior works focus on critic bias, representation-level instability in Self-Predictive Learning (SPL) under high Update-to-Data (UTD) regimes remains underexplored. To bridge this gap, we propose Robust Representation via Redundancy Reduction (R2R2), a regularization method within SPL. We theoretically identify that standard zero-centering conflicts with SPL's spectral properties and design a non-centered objective accordingly. We verify R2R2 on SPL-native algorithms like TD7. Furthermore, to demonstrate its orthogonality to prior advancements, we extend the state-of-the-art SimbaV2, which originally lacks SPL, by integrating a tailored SPL module, termed SimbaV2-SPL. Experiments across 11 continuous control tasks confirm that R2R2 effectively mitigates overfitting; specifically, at a UTD ratio of 20, it improves TD7 by ~22% and provides additional gains on top of SimbaV2-SPL, which itself establishes a new state-of-the-art. The code can be found at: https://github.com/songsang7/R2R2

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 29 of its 40 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

29 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Hyperspherical Normalization for Scalable Deep Reinforcement Learning21 Feb
  2. 2024MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL11 Oct
  3. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control25 May
  4. 2024Bridging State and History Representations: Understanding Self-Predictive RL17 Jan

Other references · 25

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2016Layer Normalizationcited by 5,669
  6. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2015Continuous control with deep reinforcement learningcited by 3,407
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  10. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  11. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  12. 2016OpenAI Gymcited by 1,736
13 more
  1. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  2. 2018DeepMind Control Suitecited by 956
  3. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  4. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  5. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  6. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  7. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  8. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  9. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
  10. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  11. 2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 95
  12. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 75
  13. 2022Understanding Self-Predictive Learning for Reinforcement Learningcited by 44

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentation
  2. 2022Representation Learning in Deep RL via Discrete Information Bottleneck
  3. 2024Symmetric Reinforcement Learning Loss for Robust Learning on Diverse Tasks and Model Scales
  4. 2021Instabilities of Offline RL with Pre-Trained Neural Representation
  5. 2026Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control
  6. 2026Squeezing More from the Stream : Learning Representation Online for Streaming Reinforcement Learning
  7. 2024Ensemble Successor Representations for Task Generalization in Offline-to-Online Reinforcement Learning
  8. 2020Critic Regularized Regression