# Redesigning Multi-Scale Neural Network for Crowd Counting

arXiv: https://arxiv.org/abs/2208.02894 · HTML: https://mlsanity.com/paper/2208.02894/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 79 references that are in the map.
- First posted on arXiv 4 Aug 2022.

## Builds on (most cited first, 30 of 47)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [Learning Factored Representations in a Deep Mixture of Experts](https://mlsanity.com/paper/1312.4314/) (2013)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization](https://mlsanity.com/paper/2001.03360/) (2020)
- [Deep Image Retrieval: Learning global representations for image search](https://mlsanity.com/paper/1604.01325/) (2016)
- [Distribution Matching for Crowd Counting](https://mlsanity.com/paper/2009.13077/) (2020)
- [JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method](https://mlsanity.com/paper/2004.03597/) (2020)
- [Switching Convolutional Neural Network for Crowd Counting](https://mlsanity.com/paper/1708.00199/) (2017)
- [Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detection](https://mlsanity.com/paper/1906.07538/) (2019)
- [Point in, Box out: Beyond Counting Persons in Crowds](https://mlsanity.com/paper/1904.01333/) (2019)
- [Leveraging Unlabeled Data for Crowd Counting by Learning to Rank](https://mlsanity.com/paper/1803.03095/) (2018)
- [Generating High-Quality Crowd Density Maps using Contextual Pyramid CNNs](https://mlsanity.com/paper/1708.00953/) (2017)
- [Perspective-Guided Convolution Networks for Crowd Counting](https://mlsanity.com/paper/1909.06966/) (2019)
- [Rethinking Spatial Invariance of Convolutional Networks for Object Counting](https://mlsanity.com/paper/2206.05253/) (2022)
- [CrowdNet: A Deep Convolutional Network for Dense Crowd Counting](https://mlsanity.com/paper/1608.06197/) (2016)
- [Exploiting Unlabeled Data in CNNs by Self-supervised Learning to Rank](https://mlsanity.com/paper/1902.06285/) (2019)
- [Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting](https://mlsanity.com/paper/2012.04529/) (2020)
- [DecideNet: Counting Varying Density Crowds Through Attention Guided Detection and Density Estimation](https://mlsanity.com/paper/1712.06679/) (2017)
- [Revisiting Perspective Information for Efficient Crowd Counting](https://mlsanity.com/paper/1807.01989/) (2018)
- [Iterative Crowd Counting](https://mlsanity.com/paper/1807.09959/) (2018)
- [From Open Set to Closed Set: Counting Objects by Spatial Divide-and-Conquer](https://mlsanity.com/paper/1908.06473/) (2019)
- [Crowd Counting with Deep Structured Scale Integration Network](https://mlsanity.com/paper/1908.08692/) (2019)
- [Learning Spatial Awareness to Improve Crowd Counting](https://mlsanity.com/paper/1909.07057/) (2019)
- [Counting with Focus for Free](https://mlsanity.com/paper/1903.12206/) (2019)
- [Learn to Scale: Generating Multipolar Normalized Density Maps for Crowd Counting](https://mlsanity.com/paper/1907.12428/) (2019)

## Built on by (most cited first, 17 of 17)

- [STEERER: Resolving Scale Variations for Counting and Localization via Selective Inheritance Learning](https://mlsanity.com/paper/2308.10468/) (2023)
- [Improving Point-based Crowd Counting and Localization Based on Auxiliary Point Guidance](https://mlsanity.com/paper/2405.10589/) (2024)
- [Domain-General Crowd Counting in Unseen Scenarios](https://mlsanity.com/paper/2212.02573/) (2022)
- [Mixture of Experts (MoE): A Big Data Perspective](https://mlsanity.com/paper/2501.16352/) (2025)
- [CountFormer: Multi-View Crowd Counting Transformer](https://mlsanity.com/paper/2407.02047/) (2024)
- [Learning Discriminative Features for Crowd Counting](https://mlsanity.com/paper/2311.04509/) (2023)
- [TreeFormer: a Semi-Supervised Transformer-based Framework for Tree Counting from a Single High Resolution Image](https://mlsanity.com/paper/2307.06118/) (2023)
- [Progressive Multi-resolution Loss for Crowd Counting](https://mlsanity.com/paper/2212.04127/) (2022)
- [Video Individual Counting for Moving Drones](https://mlsanity.com/paper/2503.10701/) (2025)
- [Boosting Adverse Weather Crowd Counting via Multi-queue Contrastive Learning](https://mlsanity.com/paper/2408.05956/) (2024)
- [Efficient Masked AutoEncoder for Video Object Counting and A Large-Scale Benchmark](https://mlsanity.com/paper/2411.13056/) (2024)
- [L2HCount:Generalizing Crowd Counting from Low to High Crowd Density via Density Simulation](https://mlsanity.com/paper/2503.12935/) (2025)
- [ProgRoCC: A Progressive Approach to Rough Crowd Counting](https://mlsanity.com/paper/2504.13405/) (2025)
- [Exploring Scale Shift in Crowd Localization under the Context of Domain Generalization](https://mlsanity.com/paper/2510.19330/) (2025)
- [Video Individual Counting and Tracking from Moving Drones: A Benchmark and Methods](https://mlsanity.com/paper/2601.12500/) (2026)
- [Generative Adversarial Perturbations with Cross-paradigm Transferability on Localized Crowd Counting](https://mlsanity.com/paper/2603.24821/) (2026)
- [Count Anything](https://mlsanity.com/paper/2605.30846/) (2026)

## Abstract (arXiv)

Perspective distortions and crowd variations make crowd counting a challenging task in computer vision. To tackle it, many previous works have used multi-scale architecture in deep neural networks (DNNs). Multi-scale branches can be either directly merged (e.g. by concatenation) or merged through the guidance of proxies (e.g. attentions) in the DNNs. Despite their prevalence, these combination methods are not sophisticated enough to deal with the per-pixel performance discrepancy over multi-scale density maps. In this work, we redesign the multi-scale neural network by introducing a hierarchical mixture of density experts, which hierarchically merges multi-scale density maps for crowd counting. Within the hierarchical structure, an expert competition and collaboration scheme is presented to encourage contributions from all scales; pixel-wise soft gating nets are introduced to provide pixel-wise soft weights for scale combinations in different hierarchies. The network is optimized using both the crowd density map and the local counting map, where the latter is obtained by local integration on the former. Optimizing both can be problematic because of their potential conflicts. We introduce a new relative local counting loss based on relative count differences among hard-predicted local regions in an image, which proves to be complementary to the conventional absolute error loss on the density map. Experiments show that our method achieves the state-of-the-art performance on five public datasets, i.e. ShanghaiTech, UCF_CC_50, JHU-CROWD++, NWPU-Crowd and Trancos.