mlsanityData to 14 Oct 2026

Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning

arXiv 1803.09956 ↗First posted 27 Mar 2018cs.RO cs.AI cs.CVcited by 128

Not on the map. Why?

Abstract (arXiv)

Skilled robotic manipulation benefits from complex synergies between non-prehensile (e.g. pushing) and prehensile (e.g. grasping) actions: pushing can help rearrange cluttered objects to make space for arms and fingers; likewise, grasping can help displace objects to make pushing movements more precise and collision-free. In this work, we demonstrate that it is possible to discover and learn these synergies from scratch through model-free deep reinforcement learning. Our method involves training two fully convolutional networks that map from visual observations to actions: one infers the utility of pushes for a dense pixel-wise sampling of end effector orientations and locations, while the other does the same for grasping. Both networks are trained jointly in a Q-learning framework and are entirely self-supervised by trial and error, where rewards are provided from successful grasps. In this way, our policy learns pushing motions that enable future grasps, while learning grasps that can leverage past pushes. During picking experiments in both simulation and real-world scenarios, we find that our system quickly learns complex behaviors amid challenging cases of clutter, and achieves better grasping success rates and picking efficiencies than baseline alternatives after only a few hours of training. We further demonstrate that our method is capable of generalizing to novel objects. Qualitative results (videos), code, pre-trained models, and simulation environments are available at http://vpg.cs.princeton.edu

Key facts. Cited by 128 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 2; 2021: 34; 2022: 21; 2023: 28; 2024: 19; 2025: 13; 2026: 11.

128 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
  3. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  4. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
  5. 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 218
  6. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  7. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 182
  8. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  9. 2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 156
  10. 2023Structured World Models from Human Videoscited by 147
  11. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 122
  12. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 119
38 more
  1. 2023RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understandingcited by 94
  2. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  3. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 83
  4. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  5. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 69
  6. 2023Waypoint-Based Imitation Learning for Robotic Manipulationcited by 61
  7. 2022SpeedFolding: Learning Efficient Bimanual Folding of Garmentscited by 52
  8. 2021Learning to See before Learning to Act: Visual Pre-training for Manipulationcited by 51
  9. 2022Equivariant Transporter Networkcited by 46
  10. 2021Act the Part: Learning Interaction Strategies for Articulated Object Part Discoverycited by 45
  11. 2023Ditto in the House: Building Articulation Models of Indoor Scenes through Interactive Perceptioncited by 41
  12. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  13. 2021INVIGORATE: Interactive Visual Grounding and Grasping in Cluttercited by 32
  14. 2023HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulationcited by 29
  15. 2022Towards Versatile Embodied Navigationcited by 26
  16. 2021Self-Supervised Euphemism Detection and Identification for Content Moderationcited by 25
  17. 2021Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Controlcited by 24
  18. 2025DyWA: Dynamics-adaptive World Action Model for Generalizable Non-prehensile Manipulationcited by 22
  19. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 21
  20. 2022Visuotactile Affordances for Cloth Manipulation with Local Controlcited by 21
  21. 2022Learning Deformable Object Manipulation from Expert Demonstrationscited by 19
  22. 2023Leveraging Symmetries in Pick and Placecited by 19
  23. 2021Distributionally Robust Policy Learning via Adversarial Environment Generationcited by 17
  24. 2024PIVOT-R: Primitive-Driven Waypoint-Aware World Model for Robotic Manipulationcited by 17
  25. 2023Visual Affordance Prediction for Guiding Robot Explorationcited by 14
  26. 2021Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Taskscited by 13
  27. 2023Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environmentcited by 13
  28. 2023Hundreds Guide Millions: Adaptive Offline Reinforcement Learning with Expert Guidancecited by 13
  29. 2022VesNet-RL: Simulation-based Reinforcement Learning for Real-World US Probe Navigationcited by 12
  30. 2023On Robot Grasp Learning Using Equivariant Modelscited by 12
  31. 2021Collision-Aware Target-Driven Object Grasping in Constrained Environmentscited by 11
  32. 2021REGRAD: A Large-Scale Relational Grasp Dataset for Safe and Object-Specific Robotic Grasping in Cluttercited by 11
  33. 2021Robotic Surgery With Lean Reinforcement Learningcited by 11
  34. 2021Action Priors for Large Action Spaces in Roboticscited by 10
  35. 2021IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenescited by 9
  36. 2022Coarse-to-fine Q-attention with Tree Expansioncited by 9
  37. 2023SCENEREPLICA: Benchmarking Real-World Robot Manipulation by Creating Replicable Scenescited by 9
  38. 2025Reinforcement Learning for Large Model: A Surveycited by 9

50 most cited of 128 All 128 (JSON)