mlsanityData to 7 Oct 2026

MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistency

arXiv 2212.09948 ↗First posted 20 Dec 2022cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Masked Modeling (MM) has demonstrated widespread success in various vision challenges, by reconstructing masked visual patches. Yet, applying MM for large-scale 3D scenes remains an open problem due to the data sparsity and scene complexity. The conventional random masking paradigm used in 2D images often causes a high risk of ambiguity when recovering the masked region of 3D scenes. To this end, we propose a novel informative-preserved reconstruction, which explores local statistics to discover and preserve the representative structured points, effectively enhancing the pretext masking task for 3D scene understanding. Integrated with a progressive reconstruction manner, our method can concentrate on modeling regional geometry and enjoy less ambiguity for masked reconstruction. Besides, such scenes with progressive masking ratios can also serve to self-distill their intrinsic spatial consistency, requiring to learn the consistent representations from unmasked areas. By elegantly combining informative-preserved reconstruction on masked areas and consistency self-distillation from unmasked areas, a unified framework called MM-3DScene is yielded. We conduct comprehensive experiments on a host of downstream tasks. The consistent improvement (e.g., +6.1 [email protected] on object detection and +2.2% mIoU on semantic segmentation) demonstrates the superiority of our approach.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 66 of its 81 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

66 of 81 references.

References · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  10. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  11. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  12. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
38 more
  1. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  2. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  3. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  4. 2020Knowledge Distillation: A Surveycited by 1,578
  5. 2020Point Transformercited by 1,462
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  7. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  8. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  9. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  10. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  11. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  12. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  13. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  14. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  15. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  16. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 502
  17. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
  18. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 456
  19. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 384
  20. 2022Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Frameworkcited by 376
  21. 2019Relation-Shape Convolutional Neural Network for Point Cloud Analysiscited by 346
  22. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  23. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 318
  24. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  25. 2018SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filterscited by 261
  26. 2020PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Samplingcited by 238
  27. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 216
  28. 2021Group-Free 3D Object Detection via Transformerscited by 215
  29. 2021PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Cloudscited by 211
  30. 2020Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contextscited by 210
  31. 2020Unsupervised Point Cloud Pre-Training via Occlusion Completioncited by 186
  32. 2017SEGCloud: Semantic Segmentation of 3D Point Cloudscited by 178
  33. 2018Tangent Convolutions for Dense Prediction in 3Dcited by 173
  34. 2021Walk in the Cloud: Learning Curves for Point Clouds Shape Analysiscited by 172
  35. 2022Stratified Transformer for 3D Point Cloud Segmentationcited by 166
  36. 2021SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformercited by 162
  37. 2022Masked Discrimination for Self-Supervised Learning on Point Cloudscited by 155
  38. 2018Point Convolutional Neural Networks by Extension Operatorscited by 154

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 5; 2025: 2; 2026: 1.

12 papers.

  1. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 123
  2. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 62
  3. 2023SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Instance Segmentationcited by 44
  4. 2024Masked Image Modeling: A Surveycited by 28
  5. 2024Bringing Masked Autoencoders Explicit Contrastive Properties for Point Cloud Self-Supervised Learningcited by 16
  6. 2024A Unified Framework for 3D Scene Understandingcited by 15
  7. 2024Fine-grained Image-to-LiDAR Contrastive Distillation with Visual Foundation Modelscited by 5
  8. 2025Masked Scene Modeling: Narrowing the Gap Between Supervised and Self-Supervised Learning in 3D Scene Understandingcited by 4
  9. 2023Multi-View Representation is What You Need for Point-Cloud Pre-Trainingcited by 3
  10. 2023Generalized Robot 3D Vision-Language Model with Fast Rendering and Pre-Training Vision-Language Alignmentcited by 1
  11. 2024Exploring the Untouched Sweeps for Conflict-Aware 3D Segmentation Pretrainingcited by 1
  12. 2026PointCSP: Cross-Sample Semantic Propagation and Stability Preservation in Self-Supervised Point Cloud Learningcited by 1