# Efficient Crowd Counting via Structured Knowledge Transfer

arXiv: https://arxiv.org/abs/2003.10120 · HTML: https://mlsanity.com/paper/2003.10120/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2020.

## Built on by (most cited first, 13 of 13)

- [CLIP-Count: Towards Text-Guided Zero-Shot Object Counting](https://mlsanity.com/paper/2305.07304/) (2023)
- [Visual Tuning](https://mlsanity.com/paper/2305.06061/) (2023)
- [Vision Transformer Off-the-Shelf: A Surprising Baseline for Few-Shot Class-Agnostic Counting](https://mlsanity.com/paper/2305.04440/) (2023)
- [The Lottery Ticket Hypothesis for Self-attention in Convolutional Neural Network](https://mlsanity.com/paper/2207.07858/) (2022)
- [SQLNet: Scale-Modulated Query and Localization Network for Few-Shot Class-Agnostic Counting](https://mlsanity.com/paper/2311.10011/) (2023)
- [Road Network Guided Fine-Grained Urban Traffic Flow Inference](https://mlsanity.com/paper/2109.14251/) (2021)
- [Multi-modal Crowd Counting via Modal Emulation](https://mlsanity.com/paper/2407.19491/) (2024)
- [Counting with Adaptive Auxiliary Learning](https://mlsanity.com/paper/2203.04061/) (2022)
- [Reducing Capacity Gap in Knowledge Distillation with Review Mechanism for Crowd Counting](https://mlsanity.com/paper/2206.05475/) (2022)
- [A Unified Object Counting Network with Object Occupation Prior](https://mlsanity.com/paper/2212.14193/) (2022)
- [Remote Sensing Object Counting with Online Knowledge Learning](https://mlsanity.com/paper/2303.10318/) (2023)
- [A Lightweight Feature Fusion Architecture For Resource-Constrained Crowd Counting](https://mlsanity.com/paper/2401.05968/) (2024)
- [Real-Time Crowd Counting for Embedded Systems with Lightweight Architecture](https://mlsanity.com/paper/2510.13250/) (2025)

## Abstract (arXiv)

Crowd counting is an application-oriented task and its inference efficiency is crucial for real-world applications. However, most previous works relied on heavy backbone networks and required prohibitive run-time consumption, which would seriously restrict their deployment scopes and cause poor scalability. To liberate these crowd counting models, we propose a novel Structured Knowledge Transfer (SKT) framework, which fully exploits the structured knowledge of a well-trained teacher network to generate a lightweight but still highly effective student network. Specifically, it is integrated with two complementary transfer modules, including an Intra-Layer Pattern Transfer which sequentially distills the knowledge embedded in layer-wise features of the teacher network to guide feature learning of the student network and an Inter-Layer Relation Transfer which densely distills the cross-layer correlation knowledge of the teacher to regularize the student's feature evolutio Consequently, our student network can derive the layer-wise and cross-layer knowledge from the teacher network to learn compact yet effective features. Extensive evaluations on three benchmarks well demonstrate the effectiveness of our SKT for extensive crowd counting models. In particular, only using around 6% of the parameters and computation cost of original models, our distilled VGG-based models obtain at least 6.5× speed-up on an Nvidia 1080 GPU and even achieve state-of-the-art performance. Our code and models are available at urlhttps://github.com/HCPLab-SYSU/SKT.