mlsanityData to 14 Oct 2026

Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations

arXiv 2104.01542 ↗First posted 4 Apr 2021cs.RO cs.AI cs.CVcited by 108

Not on the map. Why?

Abstract (arXiv)

Grasp detection in clutter requires the robot to reason about the 3D scene from incomplete and noisy perception. In this work, we draw insight that 3D reconstruction and grasp learning are two intimately connected tasks, both of which require a fine-grained understanding of local geometry details. We thus propose to utilize the synergies between grasp affordance and 3D reconstruction through multi-task learning of a shared representation. Our model takes advantage of deep implicit functions, a continuous and memory-efficient representation, to enable differentiable training of both tasks. We train the model on self-supervised grasp trials data in simulation. Evaluation is conducted on a clutter removal task, where the robot clears cluttered objects by grasping them one at a time. The experimental results in simulation and on the real robot have demonstrated that the use of implicit neural representations and joint learning of grasp affordance and 3D reconstruction have led to state-of-the-art grasping results. Our method outperforms baselines by over 10% in terms of grasp success rate. Additional results and videos can be found at https://sites.google.com/view/rpl-giga2021

Key facts. Cited by 108 papers in the map (data through 14 Oct 2026). Builds on 28 of its 60 references that are in the map. First posted on arXiv 4 Apr 2021.

Builds on

28 of 60 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  6. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  7. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  8. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  9. 2020Neural Sparse Voxel Fieldscited by 959
  10. 2020Convolutional Occupancy Networkscited by 761
  11. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  12. 2018Open3D: A Modern Library for 3D Data Processingcited by 721
16 more
  1. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  2. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 333
  3. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 264
  4. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 242
  5. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
  6. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 174
  7. 2018Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approachcited by 120
  8. 2017Grasp Pose Detection in Point Cloudscited by 111
  9. 2013Data-Driven Grasp Synthesis - A Surveycited by 106
  10. 2021Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Cluttercited by 95
  11. 2018PointNetGPD: Detecting Grasp Configurations from Point Setscited by 80
  12. 2016Shape Completion Enabled Robotic Graspingcited by 79
  13. 2018Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervisioncited by 52
  14. 2016High precision grasp pose detection in dense cluttercited by 34
  15. 2019Learning Continuous 3D Reconstructions for Geometrically Aware Graspingcited by 29
  16. 2019Robust Grasp Planning Over Uncertain Shape Completionscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 26; 2023: 30; 2024: 23; 2025: 12; 2026: 12.

108 papers.

Sort:

By area:

  1. 2021Neural Fields in Visual Computing and Beyondcited by 540
  2. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 251
  3. 2022Ditto: Building Digital Twins of Articulated Objects from Interactioncited by 156
  4. 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 139
  5. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 136
  6. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 129
  7. 2022SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusioncited by 127
  8. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 108
  9. 2023GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fieldscited by 99
  10. 2021VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objectscited by 80
  11. 2024GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Graspingcited by 78
  12. 2022ShAPO: Implicit Representations for Multi-Object Shape, Appearance, and Pose Optimizationcited by 72
38 more
  1. 2022CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimationcited by 70
  2. 2022TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinementcited by 70
  3. 2022GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRFcited by 67
  4. 2023CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objectscited by 64
  5. 2022Learning Multi-Object Dynamics with Compositional Neural Radiance Fieldscited by 63
  6. 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representationscited by 61
  7. 20233D Concept Learning and Reasoning from Multi-View Imagescited by 60
  8. 2021AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactionscited by 57
  9. 2022Few-View Object Reconstruction with Unknown Categories and Camera Posescited by 53
  10. 2022SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fieldscited by 51
  11. 2022Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objectscited by 50
  12. 2024DexGraspNet 2.0: Learning Generative Dexterous Grasping in Large-scale Synthetic Cluttered Scenescited by 44
  13. 2022Neural Shape Mating: Self-Supervised Object Assembly with Adversarial Shape Priorscited by 43
  14. 2023Ditto in the House: Building Articulation Models of Indoor Scenes through Interactive Perceptioncited by 41
  15. 2023M2T2: Multi-Task Masked Transformer for Object-centric Pick and Placecited by 41
  16. 2025Diffusion Models for Robotic Manipulation: A Surveycited by 41
  17. 2025GWM: Towards Scalable Gaussian World Models for Robotic Manipulationcited by 39
  18. 2022Edge Grasp Network: A Graph-Based SE(3)-invariant Approach to Grasp Detectioncited by 35
  19. 2023Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objectscited by 35
  20. 2023D3Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Rearrangementcited by 35
  21. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  22. 2022Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrationscited by 34
  23. 2023Grasp-Anything: Large-scale Grasp Dataset from Foundation Modelscited by 33
  24. 2025GraspGen: A Diffusion-based Framework for 6-DOF Grasping with On-Generator Trainingcited by 32
  25. 2024CoA-VLA: Improving Vision-Language-Action Models via Visual-Textual Chain-of-Affordancecited by 31
  26. 2021Learning Models as Functionals of Signed-Distance Fields for Manipulation Planningcited by 29
  27. 2022K-VIL: Keypoints-based Visual Imitation Learningcited by 27
  28. 2024RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulationcited by 26
  29. 2024Language-driven Grasp Detectioncited by 26
  30. 2024DNAct: Diffusion Guided Multi-Task 3D Policy Learningcited by 23
  31. 2021Semantically Grounded Object Matching for Robust Robotic Scene Rearrangementcited by 21
  32. 2022ALTO: Alternating Latent Topologies for Implicit 3D Reconstructioncited by 21
  33. 2022Towards Scale Balanced 6-DoF Grasp Detection in Cluttered Scenescited by 21
  34. 2023Local Neural Descriptor Fields: Locally Conditioned Object Representations for Manipulationcited by 21
  35. 2024Neural Fields in Robotics: A Surveycited by 21
  36. 20233D Implicit Transporter for Temporally Consistent Keypoint Discoverycited by 20
  37. 2024GEARS: Local Geometry-aware Hand-object Interaction Synthesiscited by 20
  38. 2023FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objectscited by 19

50 most cited of 108 All 108 (JSON)