mlsanityData to 14 Oct 2026

Learning Any-View 6DoF Robotic Grasping in Cluttered Scenes via Neural Surface Rendering

arXiv 2306.07392 ↗First posted 12 Jun 2023cs.RO cs.CV cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

A significant challenge for real-world robotic manipulation is the effective 6DoF grasping of objects in cluttered scenes from any single viewpoint without the need for additional scene exploration. This work reinterprets grasping as rendering and introduces NeuGraspNet, a novel method for 6DoF grasp detection that leverages advances in neural volumetric representations and surface rendering. It encodes the interaction between a robot's end-effector and an object's surface by jointly learning to render the local object surface and learning grasping functions in a shared feature space. The approach uses global (scene-level) features for grasp generation and local (grasp-level) neural surface features for grasp evaluation. This enables effective, fully implicit 6DoF grasp quality prediction, even in partially observed scenes. NeuGraspNet operates on random viewpoints, common in mobile manipulation scenarios, and outperforms existing implicit and semi-implicit grasping methods. The real-world applicability of the method has been demonstrated with a mobile manipulator robot, grasping in open, cluttered spaces. Project website at https://sites.google.com/view/neugraspnet

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 63 of its 86 references that are in the map. First posted on arXiv 12 Jun 2023.

Builds on

63 of 86 references.

References · 63

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  4. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  6. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  7. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  8. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  9. 2020Convolutional Occupancy Networkscited by 761
  10. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
  11. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  12. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 646
38 more
  1. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  2. 2021Vector Neurons: A General Framework for SO(3)-Equivariant Networkscited by 305
  3. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 264
  4. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 242
  5. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
  6. 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 218
  7. 2022AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domainscited by 208
  8. 2019A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithmscited by 142
  9. 2021Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objectscited by 119
  10. 2021D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactionscited by 113
  11. 2017Grasp Pose Detection in Point Cloudscited by 111
  12. 2021Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representationscited by 108
  13. 2013Data-Driven Grasp Synthesis - A Surveycited by 106
  14. 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
  15. 2015Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocolscited by 99
  16. 2021Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Cluttercited by 95
  17. 2022POCO: Point Convolution for Surface Reconstructioncited by 92
  18. 2018PointNetGPD: Detecting Grasp Configurations from Point Setscited by 80
  19. 2022Deep Learning Approaches to Grasp Synthesis: A Reviewcited by 72
  20. 2019KETO: Learning Keypoint Representations for Tool Manipulationcited by 69
  21. 2022GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRFcited by 67
  22. 2020EGAD! an Evolved Grasping Analysis Dataset for diversity and reproducibility in robotic manipulationcited by 61
  23. 2023FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Modelscited by 47
  24. 2020PointNet++ Grasping: Learning An End-to-end Spatial Grasp Generation Algorithm from Sparse Point Cloudscited by 39
  25. 2021RGB Matters: Learning 7-DoF Grasp Poses on Monocular RGBD Imagescited by 39
  26. 2022Edge Grasp Network: A Graph-Based SE(3)-invariant Approach to Grasp Detectioncited by 35
  27. 2016High precision grasp pose detection in dense cluttercited by 34
  28. 2021NeRP: Neural Rearrangement Planning for Unknown Objectscited by 34
  29. 2022Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrationscited by 34
  30. 2019Learning Task-Oriented Grasping from Human Activity Datasetscited by 32
  31. 2022Neural Grasp Distance Fields for Robot Manipulationcited by 31
  32. 2021DDGC: Generative Deep Dexterous Grasping in Cluttercited by 29
  33. 2020Same Object, Different Grasps: Data and Semantic Knowledge for Task-Oriented Graspingcited by 27
  34. 2019Learning Grasp Affordance Reasoning through Semantic Relationscited by 23
  35. 2020Grasp Proposal Networks: An End-to-End Solution for Visual Learning of Robotic Graspscited by 22
  36. 2022Robot Learning of Mobile Manipulation with Reachability Behavior Priorscited by 22
  37. 2022Contact2Grasp: 3D Grasp Synthesis via Hand-Object Contact Constraintcited by 21
  38. 2022ALTO: Alternating Latent Topologies for Implicit 3D Reconstructioncited by 21

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 2.

5 papers.

  1. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  2. 2024An Economic Framework for 6-DoF Grasp Detectioncited by 23
  3. 2024AGILE: Approach-based Grasp Inference Learned from Element Decompositioncited by 2
  4. 2025Equivariant Volumetric Graspingcited by 1
  5. 2025SpikeGrasp: A Benchmark for 6-DoF Grasp Pose Detection from Stereo Spike Streamscited by 0