# SceneComplete: Open-World 3D Scene Completion in Cluttered Real World Environments for Robot Manipulation

arXiv: https://arxiv.org/abs/2410.23643 · HTML: https://mlsanity.com/paper/2410.23643/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2024.

## Built on by (most cited first, 18 of 18)

- [Any6D: Model-free 6D Pose Estimation of Novel Objects](https://mlsanity.com/paper/2503.18673/) (2025)
- [Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modeling](https://mlsanity.com/paper/2411.19492/) (2024)
- [Reconstruction by Generation: 3D Multi-Object Scene Reconstruction from Sparse Observations](https://mlsanity.com/paper/2604.27106/) (2026)
- [Picasso: Holistic Scene Reconstruction with Physics-Constrained Sampling](https://mlsanity.com/paper/2602.08058/) (2026)
- [Simulation-Ready Cluttered Scene Estimation via Physics-aware Joint Shape and Pose Optimization](https://mlsanity.com/paper/2602.20150/) (2026)
- [Seeing Through Clutter: Structured 3D Scene Reconstruction via Iterative Object Removal](https://mlsanity.com/paper/2602.04053/) (2026)
- [Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling](https://mlsanity.com/paper/2608.30821/) (2026)
- [Object Reconstruction under Occlusion with Generative Priors and Contact-induced Constraints](https://mlsanity.com/paper/2512.05079/) (2025)
- [FlowTouch: View-Invariant Visuo-Tactile Prediction](https://mlsanity.com/paper/2603.08255/) (2026)
- [GraspFoM: Towards Reconstruction-Driven Robotic Grasping with 3D Foundation Priors](https://mlsanity.com/paper/2606.08440/) (2026)
- [Vector Quantized Feature Fields for Fast 3D Semantic Lifting](https://mlsanity.com/paper/2503.06469/) (2025)
- [Rational Inverse Reasoning: Few-Shot Imitation by Inferring Intent through Planning](https://mlsanity.com/paper/2508.08983/) (2025)
- [Object Pose and Shape Estimation for Grasping: Does it Work?](https://mlsanity.com/paper/2605.26944/) (2026)
- [Can Single-View Mesh Reconstruction Generalize to Robot Camera Rotation?](https://mlsanity.com/paper/2606.22987/) (2026)
- [CODA: Depth-Aligned Scene Completion and Object Decomposition from a Single RGB-D Image](https://mlsanity.com/paper/2609.25654/) (2026)
- [Test-Time Spatial Reasoning for Robot Manipulation Using Generative Real-to-Sim](https://mlsanity.com/paper/2609.33982/) (2026)
- [ZeroBot: Learning from Scratch in Minutes with Generative Real2Sim](https://mlsanity.com/paper/2609.34010/) (2026)
- [I2CD: Direct Image-to-Convex Decomposition for Simulation-Ready Collision Geometry](https://mlsanity.com/paper/2610.03453/) (2026)

## Abstract (arXiv)

Careful robot manipulation in every-day cluttered environments requires an accurate understanding of the 3D scene, in order to grasp and place objects stably and reliably and to avoid colliding with other objects. In general, we must construct such a 3D interpretation of a complex scene based on limited input, such as a single RGB-D image. We describe SceneComplete, a system for constructing a complete, segmented, 3D model of a scene from a single view. SceneComplete is a novel pipeline for composing general-purpose pretrained perception modules (vision-language, segmentation, image-inpainting, image-to-3D, visual-descriptors and pose-estimation) to obtain highly accurate results. We demonstrate its accuracy and effectiveness with respect to ground-truth models in a large benchmark dataset and show that its accurate whole-object reconstruction enables robust grasp proposal generation, including for a dexterous hand. We release the code and additional results on our website.