# Self-positioning Point-based Transformer for Point Cloud Understanding

arXiv: https://arxiv.org/abs/2303.16450 · HTML: https://mlsanity.com/paper/2303.16450/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 65 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 30 of 52)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies](https://mlsanity.com/paper/2206.04670/) (2022)
- [Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework](https://mlsanity.com/paper/2202.07123/) (2022)
- [Relation-Shape Convolutional Neural Network for Point Cloud Analysis](https://mlsanity.com/paper/1904.07601/) (2019)

## Built on by (most cited first, 30 of 40)

- [Point Cloud Mamba: Point Cloud Learning via State Space Model](https://mlsanity.com/paper/2403.00762/) (2024)
- [Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Model](https://mlsanity.com/paper/2404.14966/) (2024)
- [KPConvX: Modernizing Kernel Point Convolution with Kernel Attention](https://mlsanity.com/paper/2405.13194/) (2024)
- [Taylor Series-Inspired Local Structure Fitting Network for Few-shot Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2504.02454/) (2025)
- [Decoupled Local Aggregation for Point Cloud Learning](https://mlsanity.com/paper/2308.16532/) (2023)
- [GPSFormer: A Global Perception and Local Structure Fitting-based Transformer for Point Cloud Understanding](https://mlsanity.com/paper/2407.13519/) (2024)
- [MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining](https://mlsanity.com/paper/2410.00871/) (2024)
- [X-3D: Explicit 3D Structure Modeling for Point Cloud Recognition](https://mlsanity.com/paper/2404.15010/) (2024)
- [Density-guided Translator Boosts Synthetic-to-Real Unsupervised Domain Adaptive Segmentation of 3D Point Clouds](https://mlsanity.com/paper/2403.18469/) (2024)
- [Point-PRC: A Prompt Learning Based Regulation Framework for Generalizable Point Cloud Analysis](https://mlsanity.com/paper/2410.20406/) (2024)
- [UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting](https://mlsanity.com/paper/2506.09952/) (2025)
- [BFANet: Revisiting 3D Semantic Segmentation with Boundary Feature Analysis](https://mlsanity.com/paper/2503.12539/) (2025)
- [Point-Cache: Test-time Dynamic and Hierarchical Cache for Robust and Generalizable Point Cloud Analysis](https://mlsanity.com/paper/2503.12150/) (2025)
- [SeaLion: Semantic Part-Aware Latent Point Diffusion Models for 3D Generation](https://mlsanity.com/paper/2505.17721/) (2025)
- [On-the-fly Point Feature Representation for Point Clouds Analysis](https://mlsanity.com/paper/2407.21335/) (2024)
- [SP2T: Sparse Proxy Attention for Dual-stream Point Transformer](https://mlsanity.com/paper/2412.11540/) (2024)
- [KAN or MLP? Point Cloud Shows the Way Forward](https://mlsanity.com/paper/2504.13593/) (2025)
- [Rethinking Attention Module Design for Point Cloud Analysis](https://mlsanity.com/paper/2407.19294/) (2024)
- [PRISM: Periodic Representation with multIscale and Similarity graph Modelling for enhanced crystal structure property prediction](https://mlsanity.com/paper/2511.20362/) (2025)
- [PointTPA: Dynamic Network Parameter Adaptation for 3D Scene Understanding](https://mlsanity.com/paper/2604.04933/) (2026)
- [PointGS: Semantic-Consistent Unsupervised 3D Point Cloud Segmentation with 3D Gaussian Splatting](https://mlsanity.com/paper/2605.11520/) (2026)
- [Benchmarking Deep Learning Architectures for Urban Vegetation Point Cloud Semantic Segmentation from MLS](https://mlsanity.com/paper/2306.10274/) (2023)
- [AVS-Net: Point Sampling with Adaptive Voxel Size for 3D Scene Understanding](https://mlsanity.com/paper/2402.17521/) (2024)
- [GSTran: Joint Geometric and Semantic Coherence for Point Cloud Segmentation](https://mlsanity.com/paper/2408.11558/) (2024)
- [Generative Data Augmentation for Object Point Cloud Segmentation](https://mlsanity.com/paper/2505.17783/) (2025)
- [Ambiguity-aware Point Cloud Segmentation by Adaptive Margin Contrastive Learning](https://mlsanity.com/paper/2507.06592/) (2025)
- [Machine Learning Multiscale Interactions](https://mlsanity.com/paper/2605.25710/) (2026)
- [Active search and coverage using point-cloud reinforcement learning](https://mlsanity.com/paper/2312.11410/) (2023)
- [Joint Learning for Scattered Point Cloud Understanding with Hierarchical Self-Distillation](https://mlsanity.com/paper/2312.16902/) (2023)
- [Scan-to-BIM for As-built Roads: Automatic Road Digital Twinning from Semantically Labeled Point Cloud Data](https://mlsanity.com/paper/2406.12404/) (2024)

## Abstract (arXiv)

Transformers have shown superior performance on various computer vision tasks with their capabilities to capture long-range dependencies. Despite the success, it is challenging to directly apply Transformers on point clouds due to their quadratic cost in the number of points. In this paper, we present a Self-Positioning point-based Transformer (SPoTr), which is designed to capture both local and global shape contexts with reduced complexity. Specifically, this architecture consists of local self-attention and self-positioning point-based global cross-attention. The self-positioning points, adaptively located based on the input shape, consider both spatial and semantic information with disentangled attention to improve expressive power. With the self-positioning points, we propose a novel global cross-attention mechanism for point clouds, which improves the scalability of global self-attention by allowing the attention module to compute attention weights with only a small set of self-positioning points. Experiments show the effectiveness of SPoTr on three point cloud tasks such as shape classification, part segmentation, and scene segmentation. In particular, our proposed model achieves an accuracy gain of 2.6% over the previous best models on shape classification with ScanObjectNN. We also provide qualitative analyses to demonstrate the interpretability of self-positioning points. The code of SPoTr is available at https://github.com/mlvlab/SPoTr.