# Surface Vision Transformers: Attention-Based Modelling applied to Cortical Analysis

arXiv: https://arxiv.org/abs/2203.16414 · HTML: https://mlsanity.com/paper/2203.16414/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 74 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [A Generalization of Transformer Networks to Graphs](https://mlsanity.com/paper/2012.09699/) (2020)
- [How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers](https://mlsanity.com/paper/2106.10270/) (2021)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [End-to-End Semi-Supervised Object Detection with Soft Teacher](https://mlsanity.com/paper/2106.09018/) (2021)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation](https://mlsanity.com/paper/2102.08005/) (2021)
- [Fixing the train-test resolution discrepancy](https://mlsanity.com/paper/1906.06423/) (2019)
- [GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph](https://mlsanity.com/paper/2105.02605/) (2021)
- [TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-rays](https://mlsanity.com/paper/1801.04334/) (2018)
- [UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation](https://mlsanity.com/paper/2107.00781/) (2021)
- [ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration](https://mlsanity.com/paper/2104.06468/) (2021)
- [Learning Dynamic Graph Representation of Brain Connectome with Spatio-Temporal Attention](https://mlsanity.com/paper/2105.13495/) (2021)
- [Spherical CNNs on Unstructured Grids](https://mlsanity.com/paper/1901.02039/) (2019)

## Built on by (most cited first, 25 of 25)

- [Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Models](https://mlsanity.com/paper/2303.08440/) (2023)
- [LaB-GATr: geometric algebra transformers for large biomedical surface and volume meshes](https://mlsanity.com/paper/2403.07536/) (2024)
- [Vision Transformers in Medical Imaging: A Review](https://mlsanity.com/paper/2211.10043/) (2022)
- [The Multiscale Surface Vision Transformer](https://mlsanity.com/paper/2303.11909/) (2023)
- [Surface Analysis with Vision Transformers](https://mlsanity.com/paper/2205.15836/) (2022)
- [Spatio-Temporal Encoding of Brain Dynamics with Surface Masked Autoencoders](https://mlsanity.com/paper/2308.05474/) (2023)
- [Applications of interpretable deep learning in neuroimaging: a comprehensive review](https://mlsanity.com/paper/2406.17792/) (2024)
- [SurfGNN: A robust surface-based prediction model with interpretability for coactivation maps of spatial and cortical features](https://mlsanity.com/paper/2411.05825/) (2024)
- [Hierarchical Deep Feature Fusion and Ensemble Learning for Enhanced Brain Tumor MRI Classification](https://mlsanity.com/paper/2506.12363/) (2025)
- [Improved Multiscale Structural Mapping with Supervertex Vision Transformer for the Detection of Alzheimer's Disease Neurodegeneration](https://mlsanity.com/paper/2604.14837/) (2026)
- [Neurodevelopmental Phenotype Prediction: A State-of-the-Art Deep Learning Model](https://mlsanity.com/paper/2211.08831/) (2022)
- [Learning Cortical Anomaly through Masked Encoding for Unsupervised Heterogeneity Mapping](https://mlsanity.com/paper/2312.02762/) (2023)
- [Cortical Surface Diffusion Generative Models](https://mlsanity.com/paper/2402.04753/) (2024)
- [Surface Vision Mamba: Leveraging Bidirectional State Space Model for Efficient Spherical Manifold Representation](https://mlsanity.com/paper/2501.14679/) (2025)
- [X-SiT: Inherently Interpretable Surface Vision Transformers for Dementia Diagnosis](https://mlsanity.com/paper/2506.20267/) (2025)
- [Surface Vision Transformers: Flexible Attention-Based Modelling of Biomedical Surfaces](https://mlsanity.com/paper/2204.03408/) (2022)
- [Unsupervised Multimodal Surface Registration with Geometric Deep Learning](https://mlsanity.com/paper/2311.13022/) (2023)
- [Controllable Surface Diffusion Generative Model for Neurodevelopmental Trajectories](https://mlsanity.com/paper/2508.03706/) (2025)
- [Spherical Brownian Bridge Diffusion Models for Conditional Cortical Thickness Forecasting](https://mlsanity.com/paper/2509.08442/) (2025)
- [SurfAge-Net: A Hierarchical Surface-Based Network for Interpretable Fine-Grained Brain Age Prediction](https://mlsanity.com/paper/2602.06994/) (2026)
- [NeurIPS: Neuro-anatomical Inductive Priors for Sphere-based Brain Decoding](https://mlsanity.com/paper/2605.24993/) (2026)
- [vSV-ViT: Variable-size SuperVertex Vision Transformer for Cortical Surface Learning in Alzheimer's Disease](https://mlsanity.com/paper/2605.26514/) (2026)
- [International Transfer of Stochastic Cortical Self-Reconstruction](https://mlsanity.com/paper/2608.07092/) (2026)
- [Bridging Modalities on the Cortex: Surface-based MRI to PET Translation with a Diffusion Bridge](https://mlsanity.com/paper/2609.20147/) (2026)
- [FlatClip: Reusing Image Foundation Models for fMRI Representation Learning via Cortical Flatmaps](https://mlsanity.com/paper/2609.31204/) (2026)

## Abstract (arXiv)

The extension of convolutional neural networks (CNNs) to non-Euclidean geometries has led to multiple frameworks for studying manifolds. Many of those methods have shown design limitations resulting in poor modelling of long-range associations, as the generalisation of convolutions to irregular surfaces is non-trivial. Motivated by the success of attention-modelling in computer vision, we translate convolution-free vision transformer approaches to surface data, to introduce a domain-agnostic architecture to study any surface data projected onto a spherical manifold. Here, surface patching is achieved by representing spherical data as a sequence of triangular patches, extracted from a subdivided icosphere. A transformer model encodes the sequence of patches via successive multi-head self-attention layers while preserving the sequence resolution. We validate the performance of the proposed Surface Vision Transformer (SiT) on the task of phenotype regression from cortical surface metrics derived from the Developing Human Connectome Project (dHCP). Experiments show that the SiT generally outperforms surface CNNs, while performing comparably on registered and unregistered data. Analysis of transformer attention maps offers strong potential to characterise subtle cognitive developmental patterns.