# LatentGNN: Learning Efficient Non-local Relations for Visual Recognition

arXiv: https://arxiv.org/abs/1905.11634 · HTML: https://mlsanity.com/paper/1905.11634/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2019.

## Built on by (most cited first, 25 of 25)

- [Attention Mechanisms in Computer Vision: A Survey](https://mlsanity.com/paper/2111.07624/) (2021)
- [Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation](https://mlsanity.com/paper/2104.00308/) (2021)
- [Cross-Modal Progressive Comprehension for Referring Segmentation](https://mlsanity.com/paper/2105.07175/) (2021)
- [Is Attention Better Than Matrix Decomposition?](https://mlsanity.com/paper/2109.04553/) (2021)
- [GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformer](https://mlsanity.com/paper/2108.12630/) (2021)
- [Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysis](https://mlsanity.com/paper/2103.05558/) (2021)
- [TubeFormer-DeepLab: Video Mask Transformer](https://mlsanity.com/paper/2205.15361/) (2022)
- [Learning Semantic Correspondence with Sparse Annotations](https://mlsanity.com/paper/2208.06974/) (2022)
- [Making Vision Transformers Efficient from A Token Sparsification View](https://mlsanity.com/paper/2303.08685/) (2023)
- [Learning Implicit Temporal Alignment for Few-shot Video Classification](https://mlsanity.com/paper/2105.04823/) (2021)
- [Eliminating Gradient Conflict in Reference-based Line-Art Colorization](https://mlsanity.com/paper/2207.06095/) (2022)
- [3D Medical Point Transformer: Introducing Convolution to Attention Networks for Medical Point Cloud Analysis](https://mlsanity.com/paper/2112.04863/) (2021)
- [Bridging the Gap between Spatial and Spectral Domains: A Unified Framework for Graph Neural Networks](https://mlsanity.com/paper/2107.10234/) (2021)
- [A^2-FPN: Attention Aggregation based Feature Pyramid Network for Instance Segmentation](https://mlsanity.com/paper/2105.03186/) (2021)
- [Spatial Mixture-of-Experts](https://mlsanity.com/paper/2211.13491/) (2022)
- [Discriminative Latent Semantic Graph for Video Captioning](https://mlsanity.com/paper/2108.03662/) (2021)
- [Poly-NL: Linear Complexity Non-local Layers with Polynomials](https://mlsanity.com/paper/2107.02859/) (2021)
- [Graph Neural Networks in Computer Vision -- Architectures, Datasets and Common Approaches](https://mlsanity.com/paper/2212.10207/) (2022)
- [Transformed ROIs for Capturing Visual Transformations in Videos](https://mlsanity.com/paper/2106.03162/) (2021)
- [Framework-agnostic Semantically-aware Global Reasoning for Segmentation](https://mlsanity.com/paper/2212.03338/) (2022)
- [A2-FPN for Semantic Segmentation of Fine-Resolution Remotely Sensed Images](https://mlsanity.com/paper/2102.07997/) (2021)
- [Towards Consistency and Complementarity: A Multiview Graph Information Bottleneck Approach](https://mlsanity.com/paper/2210.05676/) (2022)
- [Sample-efficient Surrogate Model for Frequency Response of Linear PDEs using Self-Attentive Complex Polynomials](https://mlsanity.com/paper/2301.02747/) (2023)
- [Multiresolution Feature Guidance Based Transformer for Anomaly Detection](https://mlsanity.com/paper/2305.14880/) (2023)
- [Single-Channel Distance-Based Source Separation for Mobile GPU in Outdoor and Indoor Environments](https://mlsanity.com/paper/2501.03045/) (2025)

## Abstract (arXiv)

Capturing long-range dependencies in feature representations is crucial for many visual recognition tasks. Despite recent successes of deep convolutional networks, it remains challenging to model non-local context relations between visual features. A promising strategy is to model the feature context by a fully-connected graph neural network (GNN), which augments traditional convolutional features with an estimated non-local context representation. However, most GNN-based approaches require computing a dense graph affinity matrix and hence have difficulty in scaling up to tackle complex real-world visual problems. In this work, we propose an efficient and yet flexible non-local relation representation based on a novel class of graph neural networks. Our key idea is to introduce a latent space to reduce the complexity of graph, which allows us to use a low-rank representation for the graph affinity matrix and to achieve a linear complexity in computation. Extensive experimental evaluations on three major visual recognition tasks show that our method outperforms the prior works with a large margin while maintaining a low computation cost.