mlsanityData to 14 Oct 2026

Focus On What Matters: Separated Models For Visual-Based RL Generalization

arXiv 2410.10834 ↗First posted 29 Sep 2024cs.CV cs.AI cs.LGcited by 6

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

A primary challenge for visual-based Reinforcement Learning (RL) is to generalize effectively across unseen environments. Although previous studies have explored different auxiliary tasks to enhance generalization, few adopt image reconstruction due to concerns about exacerbating overfitting to task-irrelevant features during training. Perceiving the pre-eminence of image reconstruction in representation learning, we propose SMG (Separated Models for Generalization), a novel approach that exploits image reconstruction for generalization. SMG introduces two model branches to extract task-relevant and task-irrelevant representations separately from visual observations via cooperatively reconstruction. Built upon this architecture, we further emphasize the importance of task-relevant features for generalization. Specifically, SMG incorporates two additional consistency losses to guide the agent's focus toward task-relevant areas across different scenarios, thereby achieving free from overfitting. Extensive experiments in DMC demonstrate the SOTA performance of SMG in generalization, particularly excelling in video-background settings. Evaluations on robotic manipulation tasks further confirm the robustness of SMG in real-world applications.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 42 of its 47 references that are in the map. First posted on arXiv 29 Sep 2024.

Builds on

42 of 47 references.

References · 42

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2018World Modelscited by 1,513
  5. 2020Mastering Atari with Discrete World Modelscited by 1,007
  6. 2018DeepMind Control Suitecited by 956
  7. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  8. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  9. 2019On Variational Bounds of Mutual Informationcited by 559
  10. 2019Model-Based Reinforcement Learning for Ataricited by 502
  11. 2020Reinforcement Learning with Augmented Datacited by 468
  12. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
30 more
  1. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 366
  2. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  3. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  4. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  5. 2016Learning to Navigate in Complex Environmentscited by 198
  6. 2018A Study on Overfitting in Deep Reinforcement Learningcited by 174
  7. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 174
  8. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 144
  9. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 137
  10. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  11. 2020Self-Supervised Policy Adaptation during Deploymentcited by 123
  12. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  13. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  14. 2019Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learningcited by 104
  15. 2019Observational Overfitting in Reinforcement Learningcited by 91
  16. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  17. 2022Denoised MDPs: Learning World Models Better Than the World Itselfcited by 68
  18. 2021Learning Task Informed Abstractionscited by 58
  19. 2021Temporal Predictive Coding For Model-Based Planning In Latent Spacecited by 55
  20. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Modelscited by 53
  21. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  22. 2020On the model-based stochastic value gradient for continuous reinforcement learningcited by 47
  23. 2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 34
  24. 2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
  25. 2023Learning World Models with Identifiable Factorizationcited by 23
  26. 2022Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learningcited by 21
  27. 2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 18
  28. 2024Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learningcited by 17
  29. 2023Generalizable Visual Reinforcement Learning with Segment Anything Modelcited by 12
  30. 2023SeMAIL: Eliminating Distractors in Visual Imitation via Separated Modelscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 3.

6 papers.

  1. 2025DPMambaIR: All-in-One Image Restoration via Degradation-Aware Prompt State Space Modelcited by 3
  2. 2026Dreamer-CDP: Improving Reconstruction-free World Models Via Continuous Deterministic Representation Predictioncited by 2
  3. 2024Rejecting Hallucinated State Targets during Planningcited by 1
  4. 2025SegDAC: Visual Generalization in Reinforcement Learning via Dynamic Object Tokenscited by 0
  5. 2026StarWM: Self-Supervised Trained Attention Routing for Robust World Modelscited by 0
  6. 2026VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Task-Relevant Representation Decoupling for Visual Reinforcement Learning Generalization
  2. 2025Salience-Invariant Consistent Policy Learning for Generalization in Visual Reinforcement Learning
  3. 2025Good Actions Succeed, Bad Actions Generalize: A Case Study on Why RL Generalizes Better
  4. 2024Understanding What Affects the Generalization Gap in Visual Reinforcement Learning: Theory and Empirical Evidence
  5. 2024The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning
  6. 2025Pre-trained Visual Representations Generalize Where it Matters in Model-Based Reinforcement Learning
  7. 2025Dream to Generalize: Zero-Shot Model-Based Reinforcement Learning for Unseen Visual Distractions
  8. 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learning