# Multi-modal Crowd Counting via Modal Emulation

arXiv: https://arxiv.org/abs/2407.19491 · HTML: https://mlsanity.com/paper/2407.19491/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 46 references that are in the map.
- First posted on arXiv 28 Jul 2024.

## Builds on (most cited first, 30 of 30)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning](https://mlsanity.com/paper/2204.04799/) (2022)
- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds](https://mlsanity.com/paper/1808.01050/) (2018)
- [Bayesian Loss for Crowd Count Estimation with Point Supervision](https://mlsanity.com/paper/1908.03684/) (2019)
- [Visual Prompt Multi-Modal Tracking](https://mlsanity.com/paper/2303.10826/) (2023)
- [Multimodal Prompting with Missing Modalities for Visual Recognition](https://mlsanity.com/paper/2303.03369/) (2023)
- [UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders](https://mlsanity.com/paper/2004.05763/) (2020)
- [Boosting Crowd Counting via Multifaceted Attention](https://mlsanity.com/paper/2203.02636/) (2022)
- [Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06227/) (2020)
- [Cross Modal Transformer: Towards Fast and Robust 3D Object Detection](https://mlsanity.com/paper/2301.01283/) (2023)
- [Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting](https://mlsanity.com/paper/2012.04529/) (2020)
- [DecideNet: Counting Varying Density Crowds Through Attention Guided Detection and Density Estimation](https://mlsanity.com/paper/1712.06679/) (2017)
- [Crowd Counting with Deep Structured Scale Integration Network](https://mlsanity.com/paper/1908.08692/) (2019)
- [Anticipative Feature Fusion Transformer for Multi-Modal Action Anticipation](https://mlsanity.com/paper/2210.12649/) (2022)
- [Multi-Level Bottom-Top and Top-Bottom Feature Fusion for Crowd Counting](https://mlsanity.com/paper/1908.10937/) (2019)
- [Efficient Multimodal Fusion via Interactive Prompting](https://mlsanity.com/paper/2304.06306/) (2023)
- [Spatiotemporal Modeling for Crowd Counting in Videos](https://mlsanity.com/paper/1707.07890/) (2017)
- [Attentive Crowd Flow Machines](https://mlsanity.com/paper/1809.00101/) (2018)
- [Crowd Counting using Deep Recurrent Spatial-Aware Network](https://mlsanity.com/paper/1807.00601/) (2018)
- [Gramformer: Learning Crowd Counting via Graph-Modulated Transformer](https://mlsanity.com/paper/2401.03870/) (2024)
- [Semi-supervised Crowd Counting via Density Agency](https://mlsanity.com/paper/2209.02955/) (2022)
- [TAFNet: A Three-Stream Adaptive Fusion Network for RGB-T Crowd Counting](https://mlsanity.com/paper/2202.08517/) (2022)
- [Efficient Crowd Counting via Structured Knowledge Transfer](https://mlsanity.com/paper/2003.10120/) (2020)
- [Spatio-channel Attention Blocks for Cross-modal Crowd Counting](https://mlsanity.com/paper/2210.10392/) (2022)
- [Multi-modal Crowd Counting via a Broker Modality](https://mlsanity.com/paper/2407.07518/) (2024)

## Built on by (most cited first, 3 of 3)

- [Embodied Crowd Counting](https://mlsanity.com/paper/2503.08367/) (2025)
- [A Dual-Modulation Framework for RGB-T Crowd Counting via Spatially Modulated Attention and Adaptive Fusion](https://mlsanity.com/paper/2509.17079/) (2025)
- [Thermal-Only Crowd Counting with Deployment-Time Privacy Protection](https://mlsanity.com/paper/2605.17042/) (2026)

## Abstract (arXiv)

Multi-modal crowd counting is a crucial task that uses multi-modal cues to estimate the number of people in crowded scenes. To overcome the gap between different modalities, we propose a modal emulation-based two-pass multi-modal crowd-counting framework that enables efficient modal emulation, alignment, and fusion. The framework consists of two key components: a multi-modal inference pass and a cross-modal emulation pass. The former utilizes a hybrid cross-modal attention module to extract global and local information and achieve efficient multi-modal fusion. The latter uses attention prompting to coordinate different modalities and enhance multi-modal alignment. We also introduce a modality alignment module that uses an efficient modal consistency loss to align the outputs of the two passes and bridge the semantic gap between modalities. Extensive experiments on both RGB-Thermal and RGB-Depth counting datasets demonstrate its superior performance compared to previous methods. Code available at https://github.com/Mr-Monday/Multi-modal-Crowd-Counting-via-Modal-Emulation.