# Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoders

arXiv: https://arxiv.org/abs/2312.10726 · HTML: https://mlsanity.com/paper/2312.10726/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 46 references that are in the map.
- First posted on arXiv 17 Dec 2023.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework](https://mlsanity.com/paper/2202.07123/) (2022)
- [Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training](https://mlsanity.com/paper/2205.14401/) (2022)
- [Unsupervised Point Cloud Pre-Training via Occlusion Completion](https://mlsanity.com/paper/2010.01089/) (2020)
- [Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining](https://mlsanity.com/paper/2302.02318/) (2023)
- [Masked Discrimination for Self-Supervised Learning on Point Clouds](https://mlsanity.com/paper/2203.11183/) (2022)
- [Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline](https://mlsanity.com/paper/2106.05304/) (2021)
- [Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoders](https://mlsanity.com/paper/2212.06785/) (2022)
- [MVTN: Multi-View Transformation Network for 3D Shape Recognition](https://mlsanity.com/paper/2011.13244/) (2020)
- [Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?](https://mlsanity.com/paper/2212.08320/) (2022)
- [Improving Adversarial Robustness via Channel-wise Activation Suppressing](https://mlsanity.com/paper/2103.08307/) (2021)
- [P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting](https://mlsanity.com/paper/2208.02812/) (2022)
- [DeepGCNs: Making GCNs Go as Deep as CNNs](https://mlsanity.com/paper/1910.06849/) (2019)
- [Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Models](https://mlsanity.com/paper/2304.07221/) (2023)

## Built on by (most cited first, 30 of 42)

- [MambaIR: A Simple Baseline for Image Restoration with State-Space Model](https://mlsanity.com/paper/2402.15648/) (2024)
- [PointMamba: A Simple State Space Model for Point Cloud Analysis](https://mlsanity.com/paper/2402.10739/) (2024)
- [Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis](https://mlsanity.com/paper/2403.01439/) (2024)
- [Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning](https://mlsanity.com/paper/2410.08114/) (2024)
- [PCP-MAE: Learning to Predict Centers for Point Masked Autoencoders](https://mlsanity.com/paper/2408.08753/) (2024)
- [CLIP-Guided Generative Networks for Transferable Targeted Adversarial Attacks](https://mlsanity.com/paper/2407.10179/) (2024)
- [Positional Prompt Tuning for Efficient 3D Representation Learning](https://mlsanity.com/paper/2408.11567/) (2024)
- [GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model](https://mlsanity.com/paper/2505.04119/) (2025)
- [LCM: Locally Constrained Compact Point Cloud Model for Masked Point Modeling](https://mlsanity.com/paper/2405.17149/) (2024)
- [GPSFormer: A Global Perception and Local Structure Fitting-based Transformer for Point Cloud Understanding](https://mlsanity.com/paper/2407.13519/) (2024)
- [PointDico: Contrastive 3D Representation Learning Guided by Diffusion Models](https://mlsanity.com/paper/2512.08330/) (2025)
- [Large Point-to-Gaussian Model for Image-to-3D Generation](https://mlsanity.com/paper/2408.10935/) (2024)
- [PMA: Towards Parameter-Efficient Point Cloud Understanding via Point Mamba Adapter](https://mlsanity.com/paper/2505.20941/) (2025)
- [Towards More Diverse and Challenging Pre-training for Point Cloud Learning: Self-Supervised Cross Reconstruction with Decoupled Views](https://mlsanity.com/paper/2509.01250/) (2025)
- [UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting](https://mlsanity.com/paper/2506.09952/) (2025)
- [Mantis: Mamba-native Tuning is Efficient for 3D Point Cloud Foundation Models](https://mlsanity.com/paper/2605.03438/) (2026)
- [Point Cloud Mixture-of-Domain-Experts Model for 3D Self-supervised Learning](https://mlsanity.com/paper/2410.09886/) (2024)
- [CASL: Curvature-Augmented Self-supervised Learning for 3D Anomaly Detection](https://mlsanity.com/paper/2511.12909/) (2025)
- [PAVLM: Advancing Point Cloud based Affordance Understanding Via Vision-Language Model](https://mlsanity.com/paper/2410.11564/) (2024)
- [GeoMask3D: Geometrically Informed Mask Selection for Self-Supervised Point Cloud Learning in 3D](https://mlsanity.com/paper/2405.12419/) (2024)
- [Harnessing Text-to-Image Diffusion Models for Point Cloud Self-Supervised Learning](https://mlsanity.com/paper/2507.09102/) (2025)
- [Pre-training Point Cloud Compact Model with Partial-aware Reconstruction](https://mlsanity.com/paper/2407.09344/) (2024)
- [Masked Clustering Prediction for Unsupervised Point Cloud Pre-training](https://mlsanity.com/paper/2508.08910/) (2025)
- [PointTPA: Dynamic Network Parameter Adaptation for 3D Scene Understanding](https://mlsanity.com/paper/2604.04933/) (2026)
- [Semantic-Free Procedural 3D Shapes Are Surprisingly Good Teachers](https://mlsanity.com/paper/2411.17467/) (2024)
- [Learning from Random Subspace Exploration: Generalized Test-Time Augmentation with Self-supervised Distillation](https://mlsanity.com/paper/2507.01347/) (2025)
- [DAP-MAE: Domain-Adaptive Point Cloud Masked Autoencoder for Effective Cross-Domain Learning](https://mlsanity.com/paper/2510.21635/) (2025)
- [SSA3D: Text-Conditioned Assisted Self-Supervised Framework for Automatic Dental Abutment Design](https://mlsanity.com/paper/2512.11507/) (2025)
- [Point-SRA: Self-Representation Alignment for 3D Representation Learning](https://mlsanity.com/paper/2601.01746/) (2026)
- [Rethinking Point Clouds as Sequences: A Causal Next-Token Predictive Learning Framework](https://mlsanity.com/paper/2605.17566/) (2026)

## Abstract (arXiv)

Learning 3D representation plays a critical role in masked autoencoder (MAE) based pre-training methods for point cloud, including single-modal and cross-modal based MAE. Specifically, although cross-modal MAE methods learn strong 3D representations via the auxiliary of other modal knowledge, they often suffer from heavy computational burdens and heavily rely on massive cross-modal data pairs that are often unavailable, which hinders their applications in practice. Instead, single-modal methods with solely point clouds as input are preferred in real applications due to their simplicity and efficiency. However, such methods easily suffer from limited 3D representations with global random mask input. To learn compact 3D representations, we propose a simple yet effective Point Feature Enhancement Masked Autoencoders (Point-FEMAE), which mainly consists of a global branch and a local branch to capture latent semantic features. Specifically, to learn more compact features, a share-parameter Transformer encoder is introduced to extract point features from the global and local unmasked patches obtained by global random and local block mask strategies, followed by a specific decoder to reconstruct. Meanwhile, to further enhance features in the local branch, we propose a Local Enhancement Module with local patch convolution to perceive fine-grained local context at larger scales. Our method significantly improves the pre-training efficiency compared to cross-modal alternatives, and extensive downstream experiments underscore the state-of-the-art effectiveness, particularly outperforming our baseline (Point-MAE) by 5.16%, 5.00%, and 5.04% in three variants of ScanObjectNN, respectively. The code is available at https://github.com/zyh16143998882/AAAI24-PointFEMAE.