# GraphFPN: Graph Feature Pyramid Network for Object Detection

arXiv: https://arxiv.org/abs/2108.00580 · HTML: https://mlsanity.com/paper/2108.00580/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 59 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Hierarchical Graph Representation Learning with Differentiable Pooling](https://mlsanity.com/paper/1806.08804/) (2018)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [DeepGCNs: Can GCNs Go as Deep as CNNs?](https://mlsanity.com/paper/1904.03751/) (2019)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [Graph U-Nets](https://mlsanity.com/paper/1905.05178/) (2019)
- [NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection](https://mlsanity.com/paper/1904.07392/) (2019)
- [Context Encoding for Semantic Segmentation](https://mlsanity.com/paper/1803.08904/) (2018)
- [Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network](https://mlsanity.com/paper/1703.02719/) (2017)
- [Hierarchical Multi-Scale Attention for Semantic Segmentation](https://mlsanity.com/paper/2005.10821/) (2020)
- [Scale-Aware Trident Networks for Object Detection](https://mlsanity.com/paper/1901.01892/) (2019)

## Built on by (most cited first, 25 of 25)

- [A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective](https://mlsanity.com/paper/2209.13232/) (2022)
- [Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning](https://mlsanity.com/paper/2203.09064/) (2022)
- [AFPN: Asymptotic Feature Pyramid Network for Object Detection](https://mlsanity.com/paper/2306.15988/) (2023)
- [Tri2-plane: Thinking Head Avatar via Feature Pyramid](https://mlsanity.com/paper/2401.09386/) (2024)
- [Semantic-aligned Fusion Transformer for One-shot Object Detection](https://mlsanity.com/paper/2203.09093/) (2022)
- [You Should Look at All Objects](https://mlsanity.com/paper/2207.07889/) (2022)
- [Towards Efficient Use of Multi-Scale Features in Transformer-Based Object Detectors](https://mlsanity.com/paper/2208.11356/) (2022)
- [PVG: Progressive Vision Graph for Vision Recognition](https://mlsanity.com/paper/2308.00574/) (2023)
- [Butter: Frequency Consistency and Hierarchical Fusion for Autonomous Driving Object Detection](https://mlsanity.com/paper/2507.13373/) (2025)
- [SelectionConv: Convolutional Neural Networks for Non-rectilinear Image Data](https://mlsanity.com/paper/2207.08979/) (2022)
- [Improving Semantic Segmentation in Transformers using Hierarchical Inter-Level Attention](https://mlsanity.com/paper/2207.02126/) (2022)
- [Fusion Flow-enhanced Graph Pooling Residual Networks for Unmanned Aerial Vehicles Surveillance in Day and Night Dual Visions](https://mlsanity.com/paper/2407.12647/) (2024)
- [Learning Hierarchical Graph Representation for Image Manipulation Detection](https://mlsanity.com/paper/2201.05730/) (2022)
- [HPGNN: Using Hierarchical Graph Neural Networks for Outdoor Point Cloud Processing](https://mlsanity.com/paper/2206.02153/) (2022)
- [Dynamic Graph Induced Contour-aware Heat Conduction Network for Event-based Object Detection](https://mlsanity.com/paper/2505.12908/) (2025)
- [Auto-Focus Contrastive Learning for Image Manipulation Detection](https://mlsanity.com/paper/2211.10922/) (2022)
- [Low-light Pedestrian Detection in Visible and Infrared Image Feeds: Issues and Challenges](https://mlsanity.com/paper/2311.08557/) (2023)
- [Holistically-Nested Structure-Aware Graph Neural Network for Road Extraction](https://mlsanity.com/paper/2407.02639/) (2024)
- [Spectral Graph Reasoning Network for Hyperspectral Image Classification](https://mlsanity.com/paper/2407.02647/) (2024)
- [Graph-Boosted Attentive Network for Semantic Body Parsing](https://mlsanity.com/paper/2407.05924/) (2024)
- [KonvLiNA: Integrating Kolmogorov-Arnold Network with Linear Nystr"om Attention for feature fusion in Crop Field Detection](https://mlsanity.com/paper/2408.13160/) (2024)
- [Project-and-Fuse: Improving RGB-D Semantic Segmentation via Graph Convolution Networks](https://mlsanity.com/paper/2501.18851/) (2025)
- [MO R-CNN: Multispectral Oriented R-CNN for Object Detection in Remote Sensing Image](https://mlsanity.com/paper/2509.16957/) (2025)
- [DGA-Net: Enhancing SAM with Depth Prompting and Graph-Anchor Guidance for Camouflaged Object Detection](https://mlsanity.com/paper/2601.02831/) (2026)
- [U-shaped Multi-granularity Learning for Vision-Language Models](https://mlsanity.com/paper/2607.14966/) (2026)

## Abstract (arXiv)

Feature pyramids have been proven powerful in image understanding tasks that require multi-scale features. State-of-the-art methods for multi-scale feature learning focus on performing feature interactions across space and scales using neural networks with a fixed topology. In this paper, we propose graph feature pyramid networks that are capable of adapting their topological structures to varying intrinsic image structures and supporting simultaneous feature interactions across all scales. We first define an image-specific superpixel hierarchy for each input image to represent its intrinsic image structures. The graph feature pyramid network inherits its structure from this superpixel hierarchy. Contextual and hierarchical layers are designed to achieve feature interactions within the same scale and across different scales. To make these layers more powerful, we introduce two types of local channel attention for graph neural networks by generalizing global channel attention for convolutional neural networks. The proposed graph feature pyramid network can enhance the multiscale features from a convolutional feature pyramid network. We evaluate our graph feature pyramid network in the object detection task by integrating it into the Faster R-CNN algorithm. The modified algorithm outperforms not only previous state-of-the-art feature pyramid-based methods with a clear margin but also other popular detection methods on both MS-COCO 2017 validation and test datasets.