# 3D Medical Point Transformer: Introducing Convolution to Attention Networks for Medical Point Cloud Analysis

arXiv: https://arxiv.org/abs/2112.04863 · HTML: https://mlsanity.com/paper/2112.04863/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 66 of its 74 references that are in the map.
- First posted on arXiv 9 Dec 2021.

## Builds on (most cited first, 30 of 66)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [Efficient Content-Based Sparse Attention with Routing Transformers](https://mlsanity.com/paper/2003.05997/) (2020)
- [Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs](https://mlsanity.com/paper/1711.09869/) (2017)
- [Relation-Shape Convolutional Neural Network for Point Cloud Analysis](https://mlsanity.com/paper/1904.07601/) (2019)

## Built on by (most cited first, 23 of 23)

- [Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions](https://mlsanity.com/paper/2201.12296/) (2022)
- [Transformers in 3D Point Clouds: A Survey](https://mlsanity.com/paper/2205.07417/) (2022)
- [Deep Learning-based 3D Point Cloud Classification: A Systematic Survey and Outlook](https://mlsanity.com/paper/2311.02608/) (2023)
- [3D Vision with Transformers: A Survey](https://mlsanity.com/paper/2208.04309/) (2022)
- [RibSeg v2: A Large-scale Benchmark for Rib Labeling and Anatomical Centerline Extraction](https://mlsanity.com/paper/2210.09309/) (2022)
- [TractCloud: Registration-free tractography parcellation with a novel local-global streamline point cloud representation](https://mlsanity.com/paper/2307.09000/) (2023)
- [White Matter Tracts are Point Clouds: Neuropsychological Score Prediction and Critical Region Localization via Geometric Deep Learning](https://mlsanity.com/paper/2207.02402/) (2022)
- [Point-GN: A Non-Parametric Network Using Gaussian Positional Encoding for Point Cloud Classification](https://mlsanity.com/paper/2412.03056/) (2024)
- [PaRot: Patch-Wise Rotation-Invariant Network via Feature Disentanglement and Pose Restoration](https://mlsanity.com/paper/2302.02535/) (2023)
- [PointNeuron: 3D Neuron Reconstruction via Geometry and Topology Learning of Point Clouds](https://mlsanity.com/paper/2210.08305/) (2022)
- [Moving Frame Net: SE(3)-Equivariant Network for Volumes](https://mlsanity.com/paper/2211.03420/) (2022)
- [Rethinking Rotation Invariance with Point Cloud Registration](https://mlsanity.com/paper/2301.00149/) (2022)
- [Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey](https://mlsanity.com/paper/2402.12923/) (2024)
- [A Survey of Medical Point Cloud Shape Learning: Registration, Reconstruction and Variation](https://mlsanity.com/paper/2508.03057/) (2025)
- [Rethinking Metrics and Diffusion Architecture for 3D Point Cloud Generation](https://mlsanity.com/paper/2511.05308/) (2025)
- [3D Intracranial Aneurysm Classification and Segmentation via Unsupervised Dual-branch Learning](https://mlsanity.com/paper/2201.02198/) (2022)
- [Flow AM: Generating Point Cloud Global Explanations by Latent Alignment](https://mlsanity.com/paper/2404.18760/) (2024)
- [HFBRI-MAE: Handcrafted Feature Based Rotation-Invariant Masked Autoencoder for 3D Point Cloud Analysis](https://mlsanity.com/paper/2504.14132/) (2025)
- [TRELLIS-Enhanced Surface Features for Comprehensive Intracranial Aneurysm Analysis](https://mlsanity.com/paper/2509.03095/) (2025)
- [A Task-driven Network for Mesh Classification and Semantic Part Segmentation](https://mlsanity.com/paper/2306.05246/) (2023)
- [Frenet-Serret Frame-based Decomposition for Part Segmentation of 3D Curvilinear Structures](https://mlsanity.com/paper/2404.14435/) (2024)
- [Towards Automating the Retrospective Generation of BIM Models: A Unified Framework for 3D Semantic Reconstruction of the Built Environment](https://mlsanity.com/paper/2406.01480/) (2024)
- [Enhancing Robustness to Noise Corruption for Point Cloud Recognition via Spatial Sorting and Set-Mixing Aggregation Module](https://mlsanity.com/paper/2407.10806/) (2024)

## Abstract (arXiv)

General point clouds have been increasingly investigated for different tasks, and recently Transformer-based networks are proposed for point cloud analysis. However, there are barely related works for medical point clouds, which are important for disease detection and treatment. In this work, we propose an attention-based model specifically for medical point clouds, namely 3D medical point Transformer (3DMedPT), to examine the complex biological structures. By augmenting contextual information and summarizing local responses at query, our attention module can capture both local context and global content feature interactions. However, the insufficient training samples of medical data may lead to poor feature learning, so we apply position embeddings to learn accurate local geometry and Multi-Graph Reasoning (MGR) to examine global knowledge propagation over channel graphs to enrich feature representations. Experiments conducted on IntrA dataset proves the superiority of 3DMedPT, where we achieve the best classification and segmentation results. Furthermore, the promising generalization ability of our method is validated on general 3D point cloud benchmarks: ModelNet40 and ShapeNetPart. Code is released.