# Point Cloud Understanding via Attention-Driven Contrastive Learning

arXiv: https://arxiv.org/abs/2411.14744 · HTML: https://mlsanity.com/paper/2411.14744/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 66 references that are in the map.
- First posted on arXiv 22 Nov 2024.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework](https://mlsanity.com/paper/2202.07123/) (2022)
- [Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?](https://mlsanity.com/paper/2212.08320/) (2022)
- [Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategy](https://mlsanity.com/paper/2403.06467/) (2024)
- [Point Cloud Mamba: Point Cloud Learning via State Space Model](https://mlsanity.com/paper/2403.00762/) (2024)
- [Occupancy-MAE: Self-supervised Pre-training Large-scale LiDAR Point Clouds with Masked Occupancy Autoencoders](https://mlsanity.com/paper/2206.09900/) (2022)
- [Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Model](https://mlsanity.com/paper/2404.14966/) (2024)
- [Point-JEPA: A Joint Embedding Predictive Architecture for Self-Supervised Learning on Point Cloud](https://mlsanity.com/paper/2404.16432/) (2024)
- [PoinTramba: A Hybrid Transformer-Mamba Framework for Point Cloud Analysis](https://mlsanity.com/paper/2405.15463/) (2024)
- [Self-supervised Modal and View Invariant Feature Learning](https://mlsanity.com/paper/2005.14169/) (2020)
- [Shape2Scene: 3D Scene Representation Learning Through Pre-training on Shape Data](https://mlsanity.com/paper/2407.10200/) (2024)

## Built on by (most cited first, 1 of 1)

- [Beyond Random Masking: A Dual-Stream Approach for Rotation-Invariant Point Cloud Masked Autoencoders](https://mlsanity.com/paper/2509.14975/) (2025)

## Abstract (arXiv)

Recently Transformer-based models have advanced point cloud understanding by leveraging self-attention mechanisms, however, these methods often overlook latent information in less prominent regions, leading to increased sensitivity to perturbations and limited global comprehension. To solve this issue, we introduce PointACL, an attention-driven contrastive learning framework designed to address these limitations. Our method employs an attention-driven dynamic masking strategy that guides the model to focus on under-attended regions, enhancing the understanding of global structures within the point cloud. Then we combine the original pre-training loss with a contrastive learning loss, improving feature discrimination and generalization. Extensive experiments validate the effectiveness of PointACL, as it achieves state-of-the-art performance across a variety of 3D understanding tasks, including object classification, part segmentation, and few-shot learning. Specifically, when integrated with different Transformer backbones like Point-MAE and PointGPT, PointACL demonstrates improved performance on datasets such as ScanObjectNN, ModelNet40, and ShapeNetPart. This highlights its superior capability in capturing both global and local features, as well as its enhanced robustness against perturbations and incomplete data.