mlsanityData to 14 Oct 2026

Clutter Detection and Removal in 3D Scenes with View-Consistent Inpainting

arXiv 2304.03763 ↗First posted 7 Apr 2023cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Removing clutter from scenes is essential in many applications, ranging from privacy-concerned content filtering to data augmentation. In this work, we present an automatic system that removes clutter from 3D scenes and inpaints with coherent geometry and texture. We propose techniques for its two key components: 3D segmentation from shared properties and 3D inpainting, both of which are important problems. The definition of 3D scene clutter (frequently-moving objects) is not well captured by commonly-studied object categories in computer vision. To tackle the lack of well-defined clutter annotations, we group noisy fine-grained labels, leverage virtual rendering, and impose an instance-level area-sensitive loss. Once clutter is removed, we inpaint geometry and texture in the resulting holes by merging inpainted RGB-D images. This requires novel voting and pruning strategies that guarantee multi-view consistency across individually inpainted images for mesh reconstruction. Experiments on ScanNet and Matterport dataset show that our method outperforms baselines for clutter segmentation and 3D inpainting, both visually and quantitatively.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 40 of its 55 references that are in the map. First posted on arXiv 7 Apr 2023.

Builds on

40 of 55 references.

References · 40

  1. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  2. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  4. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  5. 2017Focal Loss for Dense Object Detectioncited by 752
  6. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 738
  7. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 670
  8. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  9. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  10. 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 246
  11. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 220
  12. 2022MAT: Mask-Aware Transformer for Large Hole Image Inpaintingcited by 218
28 more
  1. 2020A survey of loss functions for semantic segmentationcited by 217
  2. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  3. 2020Non-Local Spatial Propagation Network for Depth Completioncited by 195
  4. 2021Shape As Points: A Differentiable Poisson Solvercited by 187
  5. 2019EdgeConnect: Generative Image Inpainting with Adversarial Edge Learningcited by 163
  6. 2019Pluralistic Image Completioncited by 162
  7. 2021PENet: Towards Precise and Efficient Image Guided Depth Completioncited by 146
  8. 2022SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fieldscited by 144
  9. 2018Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Cameracited by 140
  10. 2021High-Fidelity Pluralistic Image Completion with Transformerscited by 134
  11. 2021Mix3D: Out-of-Context Data Augmentation for 3D Scenescited by 126
  12. 2019StructureFlow: Image Inpainting via Structure-aware Appearance Flowcited by 118
  13. 2019CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completioncited by 110
  14. 2020Recurrent Feature Reasoning for Image Inpaintingcited by 107
  15. 2017ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scanscited by 103
  16. 2020Virtual Multi-view Fusion for 3D Semantic Segmentationcited by 99
  17. 2021Bidirectional Projection Network for Cross Dimension Scene Understandingcited by 95
  18. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  19. 2022Removing Objects From Neural Radiance Fieldscited by 66
  20. 2021Unsupervised Depth Completion with Calibrated Backprojection Layerscited by 61
  21. 2019Dense Depth Posterior (DDP) from Single Image and Sparse Rangecited by 57
  22. 2021SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpaintingcited by 48
  23. 2020One Shot 3D Photographycited by 45
  24. 2020SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scanscited by 34
  25. 2020Project to Adapt: Domain Adaptation for Depth Completion from Noisy and Sparse Sensor Datacited by 28
  26. 2021Self-Supervised Point Cloud Completion via Inpaintingcited by 16
  27. 2022Free-form 3D Scene Inpainting with Dual-stream GANcited by 6
  28. 2018Multi-View Inpainting for RGB-D Sequencecited by 4

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 5; 2025: 2.

9 papers.

  1. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Priorcited by 56
  2. 2023PERF: Panoramic Neural Radiance Field from a Single Panoramacited by 45
  3. 2024Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpaintingcited by 24
  4. 2023Inpaint3D: 3D Scene Content Generation using 2D Inpainting Diffusioncited by 13
  5. 2024DerainNeRF: 3D Scene Estimation with Adhesive Waterdrop Removalcited by 9
  6. 2024SCINeRF: Neural Radiance Fields from a Snapshot Compressive Imagecited by 3
  7. 2025HOMER: Homography-Based Efficient Multi-view 3D Object Removalcited by 2
  8. 2024Semantic UV mapping to improve texture inpainting for indoor scenescited by 1
  9. 2025Defurnishing with X-Ray Vision: Joint Removal of Furniture from Panoramas and Meshcited by 0