# Gramformer: Learning Crowd Counting via Graph-Modulated Transformer

arXiv: https://arxiv.org/abs/2401.03870 · HTML: https://mlsanity.com/paper/2401.03870/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 53 references that are in the map.
- First posted on arXiv 8 Jan 2024.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022)
- [Rethinking Graph Transformers with Spectral Attention](https://mlsanity.com/paper/2106.03893/) (2021)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Graph Neural Networks with Learnable Structural and Positional Representations](https://mlsanity.com/paper/2110.07875/) (2021)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [Representing Long-Range Context for Graph Neural Networks with Global Attention](https://mlsanity.com/paper/2201.08821/) (2022)
- [Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds](https://mlsanity.com/paper/1808.01050/) (2018)
- [Vision GNN: An Image is Worth Graph of Nodes](https://mlsanity.com/paper/2206.00272/) (2022)
- [SCLIP: Rethinking Self-Attention for Dense Vision-Language Inference](https://mlsanity.com/paper/2312.01597/) (2023)
- [Bayesian Loss for Crowd Count Estimation with Point Supervision](https://mlsanity.com/paper/1908.03684/) (2019)
- [Rethinking Counting and Localization in Crowds:A Purely Point-Based Framework](https://mlsanity.com/paper/2107.12746/) (2021)
- [NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization](https://mlsanity.com/paper/2001.03360/) (2020)
- [GraphiT: Encoding Graph Structure in Transformers](https://mlsanity.com/paper/2106.05667/) (2021)
- [Distribution Matching for Crowd Counting](https://mlsanity.com/paper/2009.13077/) (2020)
- [JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method](https://mlsanity.com/paper/2004.03597/) (2020)
- [An End-to-End Transformer Model for Crowd Localization](https://mlsanity.com/paper/2202.13065/) (2022)
- [Boosting Crowd Counting via Multifaceted Attention](https://mlsanity.com/paper/2203.02636/) (2022)
- [Generating High-Quality Crowd Density Maps using Contextual Pyramid CNNs](https://mlsanity.com/paper/1708.00953/) (2017)
- [Perspective-Guided Convolution Networks for Crowd Counting](https://mlsanity.com/paper/1909.06966/) (2019)
- [Revisiting Perspective Information for Efficient Crowd Counting](https://mlsanity.com/paper/1807.01989/) (2018)
- [Crowd Counting with Deep Structured Scale Integration Network](https://mlsanity.com/paper/1908.08692/) (2019)
- [STEERER: Resolving Scale Variations for Counting and Localization via Selective Inheritance Learning](https://mlsanity.com/paper/2308.10468/) (2023)
- [Semi-Supervised Crowd Counting via Self-Training on Surrogate Tasks](https://mlsanity.com/paper/2007.03207/) (2020)

## Built on by (most cited first, 19 of 19)

- [Multi-modal Crowd Counting via a Broker Modality](https://mlsanity.com/paper/2407.07518/) (2024)
- [SoftHGNN: Soft Hypergraph Neural Networks for General Visual Recognition](https://mlsanity.com/paper/2505.15325/) (2025)
- [Multi-modal Crowd Counting via Modal Emulation](https://mlsanity.com/paper/2407.19491/) (2024)
- [ZIP: Scalable Crowd Counting via Zero-Inflated Poisson Modeling](https://mlsanity.com/paper/2506.19955/) (2025)
- [Boosting Adverse Weather Crowd Counting via Multi-queue Contrastive Learning](https://mlsanity.com/paper/2408.05956/) (2024)
- [Efficient Masked AutoEncoder for Video Object Counting and A Large-Scale Benchmark](https://mlsanity.com/paper/2411.13056/) (2024)
- [Depth-Assisted Network for Indiscernible Marine Object Counting with Adaptive Motion-Differentiated Feature Encoding](https://mlsanity.com/paper/2503.08152/) (2025)
- [L2HCount:Generalizing Crowd Counting from Low to High Crowd Density via Density Simulation](https://mlsanity.com/paper/2503.12935/) (2025)
- [Exploring Scale Shift in Crowd Localization under the Context of Domain Generalization](https://mlsanity.com/paper/2510.19330/) (2025)
- [2D Gaussians Spatial Transport for Point-supervised Density Regression](https://mlsanity.com/paper/2511.14477/) (2025)
- [Generative Adversarial Perturbations with Cross-paradigm Transferability on Localized Crowd Counting](https://mlsanity.com/paper/2603.24821/) (2026)
- [FocalCount: Towards Class-Count Imbalance in Class-Agnostic Counting](https://mlsanity.com/paper/2502.10677/) (2025)
- [EHNet: An Efficient Hybrid Network for Crowd Counting and Localization](https://mlsanity.com/paper/2503.12061/) (2025)
- [Local Information Matters: A Rethink of Crowd Counting](https://mlsanity.com/paper/2508.16970/) (2025)
- [A Dual-Modulation Framework for RGB-T Crowd Counting via Spatially Modulated Attention and Adaptive Fusion](https://mlsanity.com/paper/2509.17079/) (2025)
- [RepSFNet : A Single Fusion Network with Structural Reparameterization for Crowd Counting](https://mlsanity.com/paper/2601.20369/) (2026)
- [Beyond Strict Pairing: Arbitrarily Paired Training for High-Performance Infrared and Visible Image Fusion](https://mlsanity.com/paper/2603.21820/) (2026)
- [SynMVCrowd: A Large Synthetic Benchmark for Multi-view Crowd Counting and Localization](https://mlsanity.com/paper/2603.23956/) (2026)
- [A Benchmark for Semi-supervised Multi-modal Crowd Counting](https://mlsanity.com/paper/2606.03646/) (2026)

## Abstract (arXiv)

Transformer has been popular in recent crowd counting work since it breaks the limited receptive field of traditional CNNs. However, since crowd images always contain a large number of similar patches, the self-attention mechanism in Transformer tends to find a homogenized solution where the attention maps of almost all patches are identical. In this paper, we address this problem by proposing Gramformer: a graph-modulated transformer to enhance the network by adjusting the attention and input node features respectively on the basis of two different types of graphs. Firstly, an attention graph is proposed to diverse attention maps to attend to complementary information. The graph is building upon the dissimilarities between patches, modulating the attention in an anti-similarity fashion. Secondly, a feature-based centrality encoding is proposed to discover the centrality positions or importance of nodes. We encode them with a proposed centrality indices scheme to modulate the node features and similarity relationships. Extensive experiments on four challenging crowd counting datasets have validated the competitiveness of the proposed method. Code is available at {https://github.com/LoraLinH/Gramformer}.