mlsanityData to 7 Oct 2026

PatchFormer: An Efficient Point Transformer with Patch Attention

arXiv 2111.00207 ↗First posted 30 Oct 2021cs.CVcited by 43

Not on the map. Why?

Abstract (arXiv)

The point cloud learning community witnesses a modeling shift from CNNs to Transformers, where pure Transformer architectures have achieved top accuracy on the major learning benchmarks. However, existing point Transformers are computationally expensive since they need to generate a large attention map, which has quadratic complexity (both in space and time) with respect to input size. To solve this shortcoming, we introduce Patch ATtention (PAT) to adaptively learn a much smaller set of bases upon which the attention maps are computed. By a weighted summation upon these bases, PAT not only captures the global shape context but also achieves linear complexity to input size. In addition, we propose a lightweight Multi-Scale aTtention (MST) block to build attentions among features of different scales, providing the model with multi-scale features. Equipped with the PAT and MST, we construct our neural architecture called PatchFormer that integrates both modules into a joint framework for point cloud learning. Extensive experiments demonstrate that our network achieves comparable accuracy on general point cloud learning tasks with 9.2x speed-up than previous point Transformers.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 46 of its 55 references that are in the map. First posted on arXiv 30 Oct 2021.

Builds on

46 of 55 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  10. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  11. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  12. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
34 more
  1. 2020Rethinking Attention with Performerscited by 1,584
  2. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  3. 2020Point Transformercited by 1,462
  4. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  5. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  6. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  7. 2018Self-Attention with Relative Position Representationscited by 1,112
  8. 2021Video Swin Transformercited by 1,090
  9. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  10. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  11. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  12. 2020PCT: Point cloud transformercited by 730
  13. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  14. 2019Relation-Shape Convolutional Neural Network for Point Cloud Analysiscited by 346
  15. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  16. 2021PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformerscited by 284
  17. 2018SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filterscited by 261
  18. 2016Volumetric and Multi-View CNNs for Object Classification on 3D Datacited by 240
  19. 2020PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Samplingcited by 238
  20. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  21. 2018SO-Net: Self-Organizing Network for Point Cloud Analysiscited by 210
  22. 2021Rethinking and Improving Relative Position Encoding for Vision Transformercited by 206
  23. 2021Walk in the Cloud: Learning Curves for Point Clouds Shape Analysiscited by 172
  24. 2019Learning Object Bounding Boxes for 3D Instance Segmentation on Point Cloudscited by 156
  25. 2019Expectation-Maximization Attention Networks for Semantic Segmentationcited by 147
  26. 2018A2-Nets: Double Attention Networkscited by 136
  27. 2021Variational Relational Point Completion Networkcited by 103
  28. 2018Recurrent Slice Networks for 3D Segmentation of Point Cloudscited by 82
  29. 2019ACFNet: Attentional Class Feature Network for Semantic Segmentationcited by 75
  30. 2021Point Cloud Upsampling via Disentangled Refinementcited by 74
  31. 2021Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusioncited by 71
  32. 2018VoxSegNet: Volumetric CNNs for Semantic Part Segmentation of 3D Shapescited by 37
  33. 2019Linked Dynamic Graph CNN: Learning on Point Cloud via Linking Hierarchical Featurescited by 31
  34. 20143D ShapeNets: A Deep Representation for Volumetric Shapescited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 13; 2024: 10; 2025: 7; 2026: 4.

43 papers.

Sort:

By area:

  1. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 432
  2. 2023E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuningcited by 70
  3. 2023OcTr: Octree-based Transformer for 3D Object Detectioncited by 32
  4. 2023Unsupervised Deep Probabilistic Approach for Partial Point Cloud Registrationcited by 26
  5. 2022PointVector: A Vector Representation In Point Cloud Analysiscited by 21
  6. 2022LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformerscited by 21
  7. 2024Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentationcited by 20
  8. 2024Rethinking Few-shot 3D Point Cloud Semantic Segmentationcited by 18
  9. 2025LitePT: Lighter Yet Stronger Point Transformercited by 17
  10. 2024AMD: Automatic Multi-step Distillation of Large-scale Vision Modelscited by 12
  11. 2022Overlap-guided Gaussian Mixture Models for Point Cloud Registrationcited by 10
  12. 2024On permutation-invariant neural networkscited by 10
31 more
  1. 2022Improving transferability of 3D adversarial attacks with scale and shear transformationscited by 5
  2. 2022GMA3D: Local-Global Attention Learning to Estimate Occluded Motions of Scene Flowcited by 3
  3. 2022PIDS: Joint Point Interaction-Dimension Search for 3D Point Cloudcited by 3
  4. 2023SAT: Size-Aware Transformer for 3D Point Cloud Semantic Segmentationcited by 3
  5. 2023GTNet: Graph Transformer Network for 3D Point Cloud Classification and Semantic Segmentationcited by 3
  6. 2023Vision Backbone Enhancement via Multi-Stage Cross-Scale Attentioncited by 3
  7. 2025Fully-Geometric Cross-Attention for Point Cloud Registrationcited by 3
  8. 2025MSPT: Efficient Large-Scale Physical Modeling via Parallelized Multi-Scale Attentioncited by 3
  9. 2023Applying Plain Transformers to Real-World Point Cloudscited by 2
  10. 2023Exploiting Inductive Bias in Transformer for Point Cloud Classification and Segmentationcited by 2
  11. 2024Rethinking Attention Module Design for Point Cloud Analysiscited by 2
  12. 2025HydraMamba: Multi-Head State Space Model for Global Point Cloud Learningcited by 2
  13. 2022A Dual-Cycled Cross-View Transformer Network for Unified Road Layout Estimation and 3D Object Detection in the Bird's-Eye-Viewcited by 1
  14. 2023ReBotNet: Fast Real-time Video Enhancementcited by 1
  15. 2023Cloud-RAIN: Point Cloud Analysis with Reflectional Invariancecited by 1
  16. 2024Fast Kernel Scene Flowcited by 1
  17. 20243D Learnable Supertoken Transformer for LiDAR Point Cloud Scene Segmentationcited by 1
  18. 2026SPRig: Self-Supervised Pose-Invariant Rigging from Mesh Sequencescited by 1
  19. 20223DGTN: 3D Dual-Attention GLocal Transformer Network for Point Cloud Classification and Segmentationcited by 0
  20. 2023Hierarchical Point Attention for Indoor 3D Object Detectioncited by 0
  21. 2023APPT : Asymmetric Parallel Point Transformer for 3D Point Cloud Understandingcited by 0
  22. 2023VTPNet for 3D deep learning on point cloudcited by 0
  23. 2024CFPFormer: Feature-pyramid like Transformer Decoder for Segmentation and Detectioncited by 0
  24. 2024Efficient Point Transformer with Dynamic Token Aggregating for LiDAR Point Cloud Processingcited by 0
  25. 2024SWCF-Net: Similarity-weighted Convolution and Local-global Fusion for Efficient Large-scale Point Cloud Semantic Segmentationcited by 0
  26. 2025SM3D: Mitigating Spectral Bias and Semantic Dilution in Point Cloud State Space Modelscited by 0
  27. 2025BAGNet: A Boundary-Aware Graph Attention Network for 3D Point Cloud Semantic Segmentationcited by 0
  28. 2025HybridTM: Combining Transformer and Mamba for 3D Semantic Segmentationcited by 0
  29. 2026LOD-Net: Locality-Aware 3D Object Detection Using Multi-Scale Transformer Networkcited by 0
  30. 2026Invaria: Learning Scale and Density Invariance in Point Clouds via Next-Resolution Predictioncited by 0
  31. 2026PCT-Prompt: A Prompt-Guided Transformer Framework for Dense Prediction Tasks in Point Cloudscited by 0