# Combining Shape Completion and Grasp Prediction for Fast and Versatile Grasping with a Multi-Fingered Hand

arXiv: https://arxiv.org/abs/2310.20350 · HTML: https://mlsanity.com/paper/2310.20350/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 31 Oct 2023.

## Builds on (most cited first, 22 of 22)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020)
- [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019)
- [PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers](https://mlsanity.com/paper/2108.08839/) (2021)
- [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019)
- [Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis](https://mlsanity.com/paper/1612.00101/) (2016)
- [Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes](https://mlsanity.com/paper/2103.14127/) (2021)
- [SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer](https://mlsanity.com/paper/2108.04444/) (2021)
- [ShapeFormer: Transformer-based Shape Completion via Sparse Representation](https://mlsanity.com/paper/2201.10326/) (2022)
- [Shape Completion Enabled Robotic Grasping](https://mlsanity.com/paper/1609.08546/) (2016)
- [Learning 3D Shape Completion under Weak Supervision](https://mlsanity.com/paper/1805.07290/) (2018)
- [Deep Differentiable Grasp Planner for High-DOF Grippers](https://mlsanity.com/paper/2002.01530/) (2020)
- [Learning Continuous 3D Reconstructions for Geometrically Aware Grasping](https://mlsanity.com/paper/1910.00983/) (2019)
- [Learning Purely Tactile In-Hand Manipulation with a Torque-Controlled Hand](https://mlsanity.com/paper/2204.03698/) (2022)
- [Generating Grasp Poses for a High-DOF Gripper Using Neural Networks](https://mlsanity.com/paper/1903.00425/) (2019)
- [TransSC: Transformer-based Shape Completion for Grasp Evaluation](https://mlsanity.com/paper/2107.00511/) (2021)
- [Shape Completion with Prediction of Uncertain Regions](https://mlsanity.com/paper/2308.00377/) (2023)

## Built on by (most cited first, 4 of 4)

- [Composing Dextrous Grasping and In-hand Manipulation via Scoring with a Reinforcement Learning Critic](https://mlsanity.com/paper/2505.13253/) (2025)
- [You Only Scan Once: A Dynamic Scene Reconstruction Pipeline for 6-DoF Robotic Grasping of Novel Objects](https://mlsanity.com/paper/2404.03462/) (2024)
- [Evaluating Latent Generative Paradigms for High-Fidelity 3D Shape Completion from a Single Depth Image](https://mlsanity.com/paper/2511.11074/) (2025)
- [Label-Efficient Grasp Joint Prediction with Point-JEPA](https://mlsanity.com/paper/2509.13349/) (2025)

## Abstract (arXiv)

Grasping objects with limited or no prior knowledge about them is a highly relevant skill in assistive robotics. Still, in this general setting, it has remained an open problem, especially when it comes to only partial observability and versatile grasping with multi-fingered hands. We present a novel, fast, and high fidelity deep learning pipeline consisting of a shape completion module that is based on a single depth image, and followed by a grasp predictor that is based on the predicted object shape. The shape completion network is based on VQDIF and predicts spatial occupancy values at arbitrary query points. As grasp predictor, we use our two-stage architecture that first generates hand poses using an autoregressive model and then regresses finger joint configurations per pose. Critical factors turn out to be sufficient data realism and augmentation, as well as special attention to difficult cases during training. Experiments on a physical robot platform demonstrate successful grasping of a wide range of household objects based on a depth image from a single viewpoint. The whole pipeline is fast, taking only about 1 s for completing the object's shape (0.7 s) and generating 1000 grasps (0.3 s).