# A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective

arXiv: https://arxiv.org/abs/2209.13232 · HTML: https://mlsanity.com/paper/2209.13232/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 260 of its 402 references that are in the map.
- First posted on arXiv 27 Sep 2022.

## Builds on (most cited first, 30 of 260)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)

## Built on by (most cited first, 30 of 56)

- [Attending to Graph Transformers](https://mlsanity.com/paper/2302.04181/) (2023)
- [On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks](https://mlsanity.com/paper/2212.02374/) (2022)
- [Toward Visual Grounding: A Survey](https://mlsanity.com/paper/2412.20206/) (2024)
- [Graph Transformers: A Survey](https://mlsanity.com/paper/2407.09777/) (2024)
- [A Survey on Spectral Graph Neural Networks](https://mlsanity.com/paper/2302.05631/) (2023)
- [Over-Squashing in Graph Neural Networks: A Comprehensive survey](https://mlsanity.com/paper/2308.15568/) (2023)
- [Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities](https://mlsanity.com/paper/2506.18019/) (2025)
- [Mix and Reason: Reasoning over Semantic Topology with Data Mixing for Domain Generalization](https://mlsanity.com/paper/2210.07571/) (2022)
- [Rethinking Query-based Transformer for Continual Image Segmentation](https://mlsanity.com/paper/2507.07831/) (2025)
- [Continual Learning on Graphs: A Survey](https://mlsanity.com/paper/2402.06330/) (2024)
- [Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulation](https://mlsanity.com/paper/2402.07127/) (2024)
- [Graph Neural Networks in Vision-Language Image Understanding: A Survey](https://mlsanity.com/paper/2303.03761/) (2023)
- [Privacy-Preserving Graph Machine Learning from Data to Computation: A Survey](https://mlsanity.com/paper/2307.04338/) (2023)
- [Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Context](https://mlsanity.com/paper/2510.20229/) (2025)
- [MaGNAS: A Mapping-Aware Graph Neural Architecture Search Framework for Heterogeneous MPSoC Deployment](https://mlsanity.com/paper/2307.08065/) (2023)
- [Graph Neural Network for spatiotemporal data: methods and applications](https://mlsanity.com/paper/2306.00012/) (2023)
- [ClusterViG: Efficient Globally Aware Vision GNNs via Image Partitioning](https://mlsanity.com/paper/2501.10640/) (2025)
- [The Correspondence Between Bounded Graph Neural Networks and Fragments of First-Order Logic](https://mlsanity.com/paper/2505.08021/) (2025)
- [Intervene-All-Paths: Unified Mitigation of LVLM Hallucinations across Alignment Formats](https://mlsanity.com/paper/2511.17254/) (2025)
- [Sim-DETR: Unlock DETR for Temporal Sentence Grounding](https://mlsanity.com/paper/2509.23867/) (2025)
- [Closed-Loop Transfer for Weakly-supervised Affordance Grounding](https://mlsanity.com/paper/2510.17384/) (2025)
- [Inductive biases in deep learning models for weather prediction](https://mlsanity.com/paper/2304.04664/) (2023)
- [Asynchronous Events-based Panoptic Segmentation using Graph Mixer Neural Network](https://mlsanity.com/paper/2305.03640/) (2023)
- [Bridging Large Language Models and Graph Structure Learning Models for Robust Representation Learning](https://mlsanity.com/paper/2410.12096/) (2024)
- [Aggregate-Combine-Readout GNNs Are More Expressive Than Logic C2](https://mlsanity.com/paper/2508.06091/) (2025)
- [No More Sibling Rivalry: Debiasing Human-Object Interaction Detection](https://mlsanity.com/paper/2509.00760/) (2025)
- [Decoding the Surgical Scene: A Scoping Review of Scene Graphs in Surgery](https://mlsanity.com/paper/2509.20941/) (2025)
- [Learning to search for and detect objects in foveal images using deep learning](https://mlsanity.com/paper/2304.05741/) (2023)
- [GCV-Turbo: End-to-end Acceleration of GNN-based Computer Vision Tasks on FPGA](https://mlsanity.com/paper/2404.07188/) (2024)
- [Exploring Graph Mamba: A Comprehensive Survey on State-Space Models for Graph Learning](https://mlsanity.com/paper/2412.18322/) (2024)

## Abstract (arXiv)

Graph Neural Networks (GNNs) have gained momentum in graph representation learning and boosted the state of the art in a variety of areas, such as data mining (e.g., social network analysis and recommender systems), computer vision (e.g., object detection and point cloud learning), and natural language processing (e.g., relation extraction and sequence learning), to name a few. With the emergence of Transformers in natural language processing and computer vision, graph Transformers embed a graph structure into the Transformer architecture to overcome the limitations of local neighborhood aggregation while avoiding strict structural inductive biases. In this paper, we present a comprehensive review of GNNs and graph Transformers in computer vision from a task-oriented perspective. Specifically, we divide their applications in computer vision into five categories according to the modality of input data, i.e., 2D natural images, videos, 3D data, vision + language, and medical images. In each category, we further divide the applications according to a set of vision tasks. Such a task-oriented taxonomy allows us to examine how each task is tackled by different GNN-based approaches and how well these approaches perform. Based on the necessary preliminaries, we provide the definitions and challenges of the tasks, in-depth coverage of the representative approaches, as well as discussions regarding insights, limitations, and future directions.