mlsanityData to 14 Oct 2026

Merging and Disentangling Views in Visual Reinforcement Learning for Robotic Manipulation

arXiv 2505.04619 ↗First posted 7 May 2025cs.LG cs.CV cs.ROcited by 2

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Vision is well-known for its use in manipulation, especially using visual servoing. Due to the 3D nature of the world, using multiple camera views and merging them creates better representations for Q-learning and in turn, trains more sample efficient policies. Nevertheless, these multi-view policies are sensitive to failing cameras and can be burdensome to deploy. To mitigate these issues, we introduce a Merge And Disentanglement (MAD) algorithm that efficiently merges views to increase sample efficiency while simultaneously disentangling views by augmenting multi-view feature inputs with single-view features. This produces robust policies and allows lightweight deployment. We demonstrate the efficiency and robustness of our approach using Meta-World and ManiSkill3. For project website and code, see https://aalmuzairee.github.io/mad

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 61 of its 70 references that are in the map. First posted on arXiv 7 May 2025.

Builds on

61 of 70 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids27 Feb
  2. 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learning22 Jul
  3. 2024A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning27 May
  4. 2024Multi-view Disentanglement for Reinforcement Learning with Multiple Cameras22 Apr

Other references · 57

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2015Continuous control with deep reinforcement learningcited by 3,407
  7. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  8. 2020Supervised Contrastive Learningcited by 2,829
  9. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  10. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  11. 2019Contrastive Multiview Codingcited by 1,301
  12. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
38 more
  1. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  2. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  3. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  5. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  7. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 565
  8. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  9. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
  10. 2020Reinforcement Learning with Augmented Datacited by 468
  11. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  12. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  13. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
  14. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
  15. 2021Reinforcement Learning with Prototypical Representationscited by 179
  16. 2022Masked World Models for Visual Controlcited by 170
  17. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 137
  18. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 137
  19. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 117
  20. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  21. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  22. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  23. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 80
  24. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  25. 2023Imitation Bootstrapped Reinforcement Learningcited by 64
  26. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  27. 2022Vision-Based Manipulators Need to Also See from Their Handscited by 47
  28. 2023DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimizationcited by 47
  29. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 43
  30. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 41
  31. 2022X-Align: Cross-Modal Cross-View Alignment for Bird's-Eye-View Segmentationcited by 38
  32. 2021Unsupervised Visual Attention and Invariance for Reinforcement Learningcited by 30
  33. 2018Sample-Efficient Deep Reinforcement Learning via Episodic Backward Updatecited by 29
  34. 2023MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulationcited by 28
  35. 2022Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learningcited by 21
  36. 2024Privileged Sensing Scaffolds Reinforcement Learningcited by 19
  37. 2020Learning Precise 3D Manipulation from Multiple Uncalibrated Camerascited by 18
  38. 2023MoVie: Visual Model-Based Policy Adaptation for View Generalizationcited by 16

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025ManiVID-3D: Generalizable View-Invariant Reinforcement Learning for Robotic Manipulation via Disentangled 3D Representationscited by 5
  2. 2026Squint: Fast Visual Reinforcement Learning for Sim-to-Real Roboticscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Zero-Shot Visual Generalization in Robot Manipulation
  2. 2023Visual-Policy Learning through Multi-Camera View to Single-Camera View Knowledge Distillation for Robot Manipulation Tasks
  3. 2021Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations
  4. 2026Multi-Camera View Scaling for Data-Efficient Robot Imitation Learning
  5. 2025Learning to See and Act: Task-Aware Virtual View Exploration for Robotic Manipulation
  6. 2026InfiNoVA: Infinite Novel View Augmentation for Viewpoint Invariant Robot Policies
  7. 2024DEAR: Disentangled Environment and Agent Representations for Reinforcement Learning without Reconstruction
  8. 2026An Empirical Study on What Matters for Viewpoint-Generalizable Policies in Visual Imitation Learning