mlsanityData to 14 Oct 2026

Self-Predictive Dynamics for Generalization of Vision-based Reinforcement Learning

arXiv 2506.05418 ↗First posted 5 Jun 2025cs.CV cs.AI cs.LGcited by 2

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Vision-based reinforcement learning requires efficient and robust representations of image-based observations, especially when the images contain distracting (task-irrelevant) elements such as shadows, clouds, and light. It becomes more important if those distractions are not exposed during training. We design a Self-Predictive Dynamics (SPD) method to extract task-relevant features efficiently, even in unseen observations after training. SPD uses weak and strong augmentations in parallel, and learns representations by predicting inverse and forward transitions across the two-way augmented versions. In a set of MuJoCo visual control tasks and an autonomous driving task (CARLA), SPD outperforms previous studies in complex observations, and significantly improves the generalization performance for unseen observations. Our code is available at https://github.com/unigary/SPD.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 26 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

22 of 26 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  8. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  9. 2018DeepMind Control Suitecited by 956
  10. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  11. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  12. 2020Reinforcement Learning with Augmented Datacited by 468
10 more
  1. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  2. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  3. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  4. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  5. 2018The relativistic discriminator: a key element missing from standard GANcited by 238
  6. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  7. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 137
  8. 2020Self-Supervised Policy Adaptation during Deploymentcited by 123
  9. 2019Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learningcited by 104
  10. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 43

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Spectral Representation-based Reinforcement Learningcited by 3
  2. 2026V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control
  2. 2026VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning
  3. 2024Intrinsic Dynamics-Driven Generalizable Scene Representations for Vision-Oriented Decision-Making Applications
  4. 2026Mask-based Predictive Representations for Reinforcement Learning
  5. 2022Reinforcement Learning with Action-Free Pre-Training from Videos
  6. 2025Dream to Generalize: Zero-Shot Model-Based Reinforcement Learning for Unseen Visual Distractions
  7. 2024An Examination of Offline-Trained Encoders in Vision-Based Deep Reinforcement Learning for Autonomous Driving
  8. 2025Pre-trained Visual Representations Generalize Where it Matters in Model-Based Reinforcement Learning