# You Only Scan Once: A Dynamic Scene Reconstruction Pipeline for 6-DoF Robotic Grasping of Novel Objects

arXiv: https://arxiv.org/abs/2404.03462 · HTML: https://mlsanity.com/paper/2404.03462/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 31 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Builds on (most cited first, 25 of 25)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model](https://mlsanity.com/paper/2207.07115/) (2022)
- [PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers](https://mlsanity.com/paper/2108.08839/) (2021)
- [Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes](https://mlsanity.com/paper/2103.14127/) (2021)
- [BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects](https://mlsanity.com/paper/2303.14158/) (2023)
- [Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects](https://mlsanity.com/paper/2110.14217/) (2021)
- [Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations](https://mlsanity.com/paper/2104.01542/) (2021)
- [Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Clutter](https://mlsanity.com/paper/2101.01132/) (2021)
- [PointNetGPD: Detecting Grasp Configurations from Point Sets](https://mlsanity.com/paper/1809.06267/) (2018)
- [Deep Learning Approaches to Grasp Synthesis: A Review](https://mlsanity.com/paper/2207.02556/) (2022)
- [GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF](https://mlsanity.com/paper/2210.06575/) (2022)
- [PointNet++ Grasping: Learning An End-to-end Spatial Grasp Generation Algorithm from Sparse Point Clouds](https://mlsanity.com/paper/2003.09644/) (2020)
- [RGB Matters: Learning 7-DoF Grasp Poses on Monocular RGBD Images](https://mlsanity.com/paper/2103.02184/) (2021)
- [Voxel-based Network for Shape Completion by Leveraging Edge Generation](https://mlsanity.com/paper/2108.09936/) (2021)
- [Towards Scale Balanced 6-DoF Grasp Detection in Cluttered Scenes](https://mlsanity.com/paper/2212.05275/) (2022)
- [A Real2Sim2Real Method for Robust Object Grasping with Neural Surface Reconstruction](https://mlsanity.com/paper/2210.02685/) (2022)
- [Robust Grasp Planning Over Uncertain Shape Completions](https://mlsanity.com/paper/1903.00645/) (2019)
- [Simultaneous Semantic and Collision Learning for 6-DoF Grasp Pose Estimation](https://mlsanity.com/paper/2108.02425/) (2021)
- [GPR: Grasp Pose Refinement Network for Cluttered Scenes](https://mlsanity.com/paper/2105.08502/) (2021)
- [3DSGrasp: 3D Shape-Completion for Robotic Grasp](https://mlsanity.com/paper/2301.00866/) (2023)
- [Beyond Top-Grasps Through Scene Completion](https://mlsanity.com/paper/1909.12908/) (2019)
- [Volumetric-based Contact Point Detection for 7-DoF Grasping](https://mlsanity.com/paper/2209.06675/) (2022)
- [Combining Shape Completion and Grasp Prediction for Fast and Versatile Grasping with a Multi-Fingered Hand](https://mlsanity.com/paper/2310.20350/) (2023)
- [Anthropomorphic Grasping with Neural Object Shape Completion](https://mlsanity.com/paper/2311.02510/) (2023)
- [Towards Confidence-guided Shape Completion for Robotic Applications](https://mlsanity.com/paper/2209.04300/) (2022)

## Built on by (most cited first, 2 of 2)

- [GraspFoM: Towards Reconstruction-Driven Robotic Grasping with 3D Foundation Priors](https://mlsanity.com/paper/2606.08440/) (2026)
- [BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation](https://mlsanity.com/paper/2608.05042/) (2026)

## Abstract (arXiv)

In the realm of robotic grasping, achieving accurate and reliable interactions with the environment is a pivotal challenge. Traditional methods of grasp planning methods utilizing partial point clouds derived from depth image often suffer from reduced scene understanding due to occlusion, ultimately impeding their grasping accuracy. Furthermore, scene reconstruction methods have primarily relied upon static techniques, which are susceptible to environment change during manipulation process limits their efficacy in real-time grasping tasks. To address these limitations, this paper introduces a novel two-stage pipeline for dynamic scene reconstruction. In the first stage, our approach takes scene scanning as input to register each target object with mesh reconstruction and novel object pose tracking. In the second stage, pose tracking is still performed to provide object poses in real-time, enabling our approach to transform the reconstructed object point clouds back into the scene. Unlike conventional methodologies, which rely on static scene snapshots, our method continuously captures the evolving scene geometry, resulting in a comprehensive and up-to-date point cloud representation. By circumventing the constraints posed by occlusion, our method enhances the overall grasp planning process and empowers state-of-the-art 6-DoF robotic grasping algorithms to exhibit markedly improved accuracy.