mlsanityData to 7 Oct 2026

Transformers in 3D Point Clouds: A Survey

arXiv 2205.07417 ↗First posted 16 May 2022cs.CVcited by 31

Not on the map. Why?

Abstract (arXiv)

Transformers have been at the heart of the Natural Language Processing (NLP) and Computer Vision (CV) revolutions. The significant success in NLP and CV inspired exploring the use of Transformers in point cloud processing. However, how do Transformers cope with the irregularity and unordered nature of point clouds? How suitable are Transformers for different 3D representations (e.g., point- or voxel-based)? How competent are Transformers for various 3D processing tasks? As of now, there is still no systematic survey of the research on these issues. For the first time, we provided a comprehensive overview of increasingly popular Transformers for 3D point cloud analysis. We start by introducing the theory of the Transformer architecture and reviewing its applications in 2D/3D fields. Then, we present three different taxonomies (i.e., implementation-, data representation-, and task-based), which can classify current Transformer-based methods from multiple perspectives. Furthermore, we present the results of an investigation of the variants and improvements of the self-attention mechanism in 3D. To demonstrate the superiority of Transformers in point cloud analysis, we present comprehensive comparisons of various Transformer-based methods for classification, segmentation, and object detection. Finally, we suggest three potential research directions, providing benefit references for the development of 3D Transformers.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 114 of its 157 references that are in the map. First posted on arXiv 16 May 2022.

Builds on

114 of 157 references.

References · 114

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2016Densely Connected Convolutional Networkscited by 6,691
  10. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  11. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  12. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
38 more
  1. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  3. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2017Non-local Neural Networkscited by 2,290
  6. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  7. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  8. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  9. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  10. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  11. 2020Point Transformercited by 1,462
  12. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  13. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  14. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  15. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  16. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  17. 2021Transformers in Vision: A Surveycited by 1,009
  18. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  19. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  20. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  21. 2020PCT: Point cloud transformercited by 730
  22. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  23. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  24. 2020A Survey on Visual Transformercited by 653
  25. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  26. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  27. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  28. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 530
  29. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  30. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  31. 2021Transformer Trackingcited by 412
  32. 2018PCN: Point Completion Networkcited by 382
  33. 2022Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Frameworkcited by 376
  34. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  35. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  36. 20163DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructionscited by 333
  37. 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 333
  38. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329

50 most cited of 114 All 114 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 5; 2024: 8; 2025: 6; 2026: 5.

31 papers.

Sort:

  1. 2022Masked Autoencoder for Self-Supervised Pre-training on Lidar Point Cloudscited by 39
  2. 2022GLT-T: Global-Local Transformer Voting for 3D Single Object Tracking in Point Cloudscited by 23
  3. 2022CSDN: Cross-modal Shape-transfer Dual-refinement Network for Point Cloud Completioncited by 22
  4. 2023GMSF: Global Matching Scene Flowcited by 14
  5. 2024Trackastra: Transformer-based cell tracking for live-cell microscopycited by 14
  6. 2022Analyzing Deep Learning Representations of Point Clouds for Real-Time In-Vehicle LiDAR Perceptioncited by 8
  7. 2022UTOPIC: Uncertainty-aware Overlap Prediction Network for Partial Point Cloud Registrationcited by 6
  8. 2022MODNet: Multi-offset Point Cloud Denoising Network Customized for Multi-scale Patchescited by 5
  9. 2024RoMo: A Robust Solver for Full-body Unlabeled Optical Motion Capturecited by 5
  10. 2023GLT-T++: Global-Local Transformer for 3D Siamese Tracking with Ranking Losscited by 4
  11. 20233D Multiple Object Tracking on Autonomous Driving: A Literature Reviewcited by 4
  12. 2024Point Tree Transformer for Point Cloud Registrationcited by 4
19 more
  1. 2024Gridded Transformer Neural Processes for Large Unstructured Spatio-Temporal Datacited by 4
  2. 2023HybridNet: Dual-Branch Fusion of Geometrical and Topological Views for VLSI Congestion Predictioncited by 2
  3. 2024PCTreeS: 3D Point Cloud Tree Species Classification Using Airborne LiDAR Imagescited by 2
  4. 2026PointTPA: Dynamic Network Parameter Adaptation for 3D Scene Understandingcited by 2
  5. 2023Point Transformer with Federated Learning for Predicting Breast Cancer HER2 Status from Hematoxylin and Eosin-Stained Whole Slide Imagescited by 1
  6. 2025EuLearn: A 3D database for learning Euler characteristicscited by 1
  7. 2025HyperPointFormer: Multimodal Fusion in 3D Space with Dual-Branch Cross-Attention Transformerscited by 1
  8. 2025Part Segmentation of Human Meshes via Multi-View Human Parsingcited by 1
  9. 2026NPNet: A Non-Parametric Network with Adaptive Gaussian-Fourier Positional Encoding for 3D Classification and Segmentationcited by 1
  10. 2026RelFlexformer: Efficient Attention 3D-Transformers for Integrable Relative Positional Encodingscited by 1
  11. 20223DGTN: 3D Dual-Attention GLocal Transformer Network for Point Cloud Classification and Segmentationcited by 0
  12. 2024Exploiting Topological Priors for Boosting Point Cloud Generationcited by 0
  13. 2024Localized Gaussians as Self-Attention Weights for Point Clouds Correspondencecited by 0
  14. 2024Improving Adversarial Robustness for 3D Point Cloud Recognition at Test-Time through Purified Self-Trainingcited by 0
  15. 2025Relational inductive biases on attention mechanismscited by 0
  16. 2025LiLa-Net: Lightweight Latent LiDAR Autoencoder for 3D Point Cloud Reconstructioncited by 0
  17. 2025Human Action Recognition from Point Clouds over Timecited by 0
  18. 2026DRIFT: Dual-Representation Inter-Fusion Transformer for Automated Driving Perception with 4D Radar Point Cloudscited by 0
  19. 2026Bridging the Dimensionality Gap: A Taxonomy and Survey of 2D Vision Model Adaptation for 3D Analysiscited by 0