mlsanityData to 7 Oct 2026

PointGAC: Geometric-Aware Codebook for Masked Point Cloud Modeling

arXiv 2507.04801 ↗First posted 7 Jul 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Most masked point cloud modeling (MPM) methods follow a regression paradigm to reconstruct the coordinate or feature of masked regions. However, they tend to over-constrain the model to learn the details of the masked region, resulting in failure to capture generalized features. To address this limitation, we propose textbftextitPointGAC, a novel clustering-based MPM method that aims to align the feature distribution of masked regions. Specially, it features an online codebook-guided teacher-student framework. Firstly, it presents a geometry-aware partitioning strategy to extract initial patches. Then, the teacher model updates a codebook via online k-means based on features extracted from the complete patches. This procedure facilitates codebook vectors to become cluster centers. Afterward, we assigns the unmasked features to their corresponding cluster centers, and the student model aligns the assignment for the reconstructed masked features. This strategy focuses on identifying the cluster centers to which the masked features belong, enabling the model to learn more generalized feature representations. Benefiting from a proposed codebook maintenance mechanism, codebook vectors are actively updated, which further increases the efficiency of semantic feature learning. Experiments validate the effectiveness of the proposed method on various downstream tasks. Code is available at https://github.com/LAB123-tech/PointGAC

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 55 references that are in the map. First posted on arXiv 7 Jul 2025.

Builds on

42 of 55 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  12. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
30 more
  1. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  2. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  3. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  4. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 456
  5. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 384
  6. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  7. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 284
  8. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  9. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
  10. 2022Masked Discrimination for Self-Supervised Learning on Point Cloudscited by 155
  11. 2023PointGPT: Auto-regressively Generative Pre-training from Point Cloudscited by 115
  12. 2021Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samplescited by 114
  13. 2020OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learningcited by 65
  14. 2023Human Pose as Compositional Tokenscited by 45
  15. 2023Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoderscited by 42
  16. 2022Masked Autoencoders in 3D Point Cloud Representation Learningcited by 30
  17. 2024Point-JEPA: A Joint Embedding Predictive Architecture for Self-Supervised Learning on Point Cloudcited by 29
  18. 2023Point2Vec for Self-Supervised Representation Learning on Point Cloudscited by 28
  19. 2023A Conditional Denoising Diffusion Probabilistic Model for Point Cloud Upsamplingcited by 28
  20. 20223DCTN: 3D Convolution-Transformer Network for Point Cloud Classificationcited by 25
  21. 2024PCP-MAE: Learning to Predict Centers for Point Masked Autoencoderscited by 23
  22. 2024An End-to-End Robust Point Cloud Semantic Segmentation Network with Single-Step Conditional Diffusion Modelscited by 22
  23. 20223D-OAE: Occlusion Auto-Encoders for Self-Supervised Learning on Point Cloudscited by 18
  24. 2022POS-BERT: Point Cloud One-Stage BERT Pre-Trainingcited by 16
  25. 2024RI-MAE: Rotation-Invariant Masked AutoEncoders for Self-Supervised Point Cloud Representation Learningcited by 15
  26. 20243D-JEPA: A Joint Embedding Predictive Architecture for 3D Self-Supervised Representation Learningcited by 14
  27. 2025Cross-Modal and Uncertainty-Aware Agglomeration for Open-Vocabulary 3D Scene Understandingcited by 13
  28. 2022Data Augmentation-free Unsupervised Learning for 3D Point Cloud Understandingcited by 6
  29. 2023PointGame: Geometrically and Adaptively Masked Auto-Encoder on Point Cloudscited by 4
  30. 2024GSTran: Joint Geometric and Semantic Coherence for Point Cloud Segmentationcited by 1

Built on by

Not cited yet.