# 3D-CDRGP: Towards Cross-Device Robotic Grasping Policy in 3D Open World

arXiv: https://arxiv.org/abs/2411.18133 · HTML: https://mlsanity.com/paper/2411.18133/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Builds on (most cited first, 24 of 24)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Point Transformer V3: Simpler, Faster, Stronger](https://mlsanity.com/paper/2312.10035/) (2023)
- [PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation](https://mlsanity.com/paper/2004.01658/) (2020)
- [SoftGroup for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/2203.01509/) (2022)
- [Open-TeleVision: Teleoperation with Immersive Active Visual Feedback](https://mlsanity.com/paper/2407.01512/) (2024)
- [DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation](https://mlsanity.com/paper/2210.02697/) (2022)
- [Hierarchical Aggregation for 3D Instance Segmentation](https://mlsanity.com/paper/2108.02350/) (2021)
- [AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation](https://mlsanity.com/paper/2410.00371/) (2024)
- [Data-Driven Grasp Synthesis - A Survey](https://mlsanity.com/paper/1309.2660/) (2013)
- [GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving](https://mlsanity.com/paper/1903.10955/) (2019)
- [Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks](https://mlsanity.com/paper/2108.07478/) (2021)
- [SAI3D: Segment Any Instance in 3D Scenes](https://mlsanity.com/paper/2312.11557/) (2023)
- [GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Grasping](https://mlsanity.com/paper/2403.09637/) (2024)
- [DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution](https://mlsanity.com/paper/2011.13328/) (2020)
- [Sparse 3D convolutional neural networks](https://mlsanity.com/paper/1505.02890/) (2015)
- [Omnigrasp: Grasping Diverse Objects with Simulated Humanoids](https://mlsanity.com/paper/2407.11385/) (2024)
- [MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation](https://mlsanity.com/paper/2203.14662/) (2022)
- [Divide and Conquer: 3D Point Cloud Instance Segmentation With Point-Wise Binarization](https://mlsanity.com/paper/2207.11209/) (2022)
- [Challenges for Monocular 6D Object Pose Estimation in Robotics](https://mlsanity.com/paper/2307.12172/) (2023)
- [ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentation](https://mlsanity.com/paper/2210.07450/) (2022)
- [FoundationGrasp: Generalizable Task-Oriented Grasping with Foundation Models](https://mlsanity.com/paper/2404.10399/) (2024)
- [Open-Pose 3D Zero-Shot Learning: Benchmark and Challenges](https://mlsanity.com/paper/2312.07039/) (2023)

## Abstract (arXiv)

Given the diversity of devices and the product upgrades, cross-device research has become an urgent issue that needs to be tackled. To this end, we pioneer in probing the cross-device (cameras & robotics) grasping policy in the 3D open world. Specifically, we construct two real-world grasping setups, employing robotic arms and cameras from completely different manufacturers. To minimize domain differences in point clouds from diverse cameras, we adopt clustering methods to generate 3D object proposals. However, existing clustering methods are limited to closed-set scenarios, which confines the robotic graspable object categories and ossifies the deployment scenarios. To extend these methods to open-world settings, we introduce the SSGC-Seg module that enables category-agnostic 3D object detection. The proposed module transforms the original multi-class semantic information into binary semantic cues-foreground and background by analyzing the SoftMax value of each point, and then clusters the foreground points based on geometric information to form initial object proposals. Furthermore, ScoreNetddag is designed to score each detection result, and the robotic arm prioritizes grasping the object with the highest confidence score. Experiments on two different types of setups highlight the effectiveness and robustness of our policy for cross-device robotics grasping research. Our code is provided in the supplementary and will be released upon acceptance.