# A Unified Multi-Task Learning Framework of Real-Time Drone Supervision for Crowd Counting

arXiv: https://arxiv.org/abs/2202.03843 · HTML: https://mlsanity.com/paper/2202.03843/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 8 Feb 2022.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [Bayesian Loss for Crowd Count Estimation with Point Supervision](https://mlsanity.com/paper/1908.03684/) (2019)
- [Context-Aware Crowd Counting](https://mlsanity.com/paper/1811.10452/) (2018)
- [Distribution Matching for Crowd Counting](https://mlsanity.com/paper/2009.13077/) (2020)
- [RFN-Nest: An end-to-end residual fusion network for infrared and visible images](https://mlsanity.com/paper/2103.04286/) (2021)
- [DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2003.09210/) (2020)
- [NestFuse: An Infrared and Visible Image Fusion Architecture based on Nest Connection and Spatial/Channel Attention Models](https://mlsanity.com/paper/2007.00328/) (2020)
- [SCAR: Spatial-/Channel-wise Attention Regression Networks for Crowd Counting](https://mlsanity.com/paper/1908.03716/) (2019)
- [Attention-guided Context Feature Pyramid Network for Object Detection](https://mlsanity.com/paper/2005.11475/) (2020)
- [Crowd Counting via Weighted VLAD on Dense Attribute Feature Maps](https://mlsanity.com/paper/1604.08660/) (2016)
- [Dense Crowds Detection and Surveillance with Drones using Density Maps](https://mlsanity.com/paper/2003.08766/) (2020)

## Built on by (most cited first, 5 of 5)

- [Revisiting Crowd Counting: State-of-the-art, Trends, and Future Perspectives](https://mlsanity.com/paper/2209.07271/) (2022)
- [CLIP: Train Faster with Less Data](https://mlsanity.com/paper/2212.01452/) (2022)
- [DroneNet: Crowd Density Estimation using Self-ONNs for Drones](https://mlsanity.com/paper/2211.07137/) (2022)
- [Why Existing Multimodal Crowd Counting Datasets Can Lead to Unfulfilled Expectations in Real-World Applications](https://mlsanity.com/paper/2304.06401/) (2023)
- [Curriculum for Crowd Counting -- Is it Worthy?](https://mlsanity.com/paper/2401.07586/) (2024)

## Abstract (arXiv)

In this paper, a novel Unified Multi-Task Learning Framework of Real-Time Drone Supervision for Crowd Counting (MFCC) is proposed, which utilizes an image fusion network architecture to fuse images from the visible and thermal infrared image, and a crowd counting network architecture to estimate the density map. The purpose of our framework is to fuse two modalities, including visible and thermal infrared images captured by drones in real-time, that exploit the complementary information to accurately count the dense population and then automatically guide the flight of the drone to supervise the dense crowd. To this end, we propose the unified multi-task learning framework for crowd counting for the first time and re-design the unified training loss functions to align the image fusion network and crowd counting network. We also design the Assisted Learning Module (ALM) to fuse the density map feature to the image fusion encoder process for learning the counting features. To improve the accuracy, we propose the Extensive Context Extraction Module (ECEM) that is based on a dense connection architecture to encode multi-receptive-fields contextual information and apply the Multi-domain Attention Block (MAB) for concerning the head region in the drone view. Finally, we apply the prediction map to automatically guide the drones to supervise the dense crowd. The experimental results on the DroneRGBT dataset show that, compared with the existing methods, ours has comparable results on objective evaluations and an easier training process.