# Multi-modal Crowd Counting via a Broker Modality

arXiv: https://arxiv.org/abs/2407.07518 · HTML: https://mlsanity.com/paper/2407.07518/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 68 references that are in the map.
- First posted on arXiv 10 Jul 2024.

## Builds on (most cited first, 27 of 27)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds](https://mlsanity.com/paper/1808.01050/) (2018)
- [Bayesian Loss for Crowd Count Estimation with Point Supervision](https://mlsanity.com/paper/1908.03684/) (2019)
- [DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion](https://mlsanity.com/paper/2303.06840/) (2023)
- [Topological Planning with Transformers for Vision-and-Language Navigation](https://mlsanity.com/paper/2012.05292/) (2020)
- [UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders](https://mlsanity.com/paper/2004.05763/) (2020)
- [Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detection](https://mlsanity.com/paper/1906.07538/) (2019)
- [Boosting Crowd Counting via Multifaceted Attention](https://mlsanity.com/paper/2203.02636/) (2022)
- [Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06227/) (2020)
- [Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting](https://mlsanity.com/paper/2012.04529/) (2020)
- [DecideNet: Counting Varying Density Crowds Through Attention Guided Detection and Density Estimation](https://mlsanity.com/paper/1712.06679/) (2017)
- [Crowd Counting with Deep Structured Scale Integration Network](https://mlsanity.com/paper/1908.08692/) (2019)
- [Point-Query Quadtree for Crowd Counting, Localization, and More](https://mlsanity.com/paper/2308.13814/) (2023)
- [Multi-Level Bottom-Top and Top-Bottom Feature Fusion for Crowd Counting](https://mlsanity.com/paper/1908.10937/) (2019)
- [A Self-Training Approach for Point-Supervised Object Detection and Counting in Crowds](https://mlsanity.com/paper/2007.12831/) (2020)
- [Semi-Supervised Crowd Counting via Self-Training on Surrogate Tasks](https://mlsanity.com/paper/2007.03207/) (2020)
- [A Comparison of Pre-trained Vision-and-Language Models for Multimodal Representation Learning across Medical Images and Reports](https://mlsanity.com/paper/2009.01523/) (2020)
- [CommerceMM: Large-Scale Commerce MultiModal Representation Learning with Omni Retrieval](https://mlsanity.com/paper/2202.07247/) (2022)
- [Crowd Counting using Deep Recurrent Spatial-Aware Network](https://mlsanity.com/paper/1807.00601/) (2018)
- [Gramformer: Learning Crowd Counting via Graph-Modulated Transformer](https://mlsanity.com/paper/2401.03870/) (2024)
- [Direct Measure Matching for Crowd Counting](https://mlsanity.com/paper/2107.01558/) (2021)
- [Semi-supervised Crowd Counting via Density Agency](https://mlsanity.com/paper/2209.02955/) (2022)
- [TAFNet: A Three-Stream Adaptive Fusion Network for RGB-T Crowd Counting](https://mlsanity.com/paper/2202.08517/) (2022)
- [Spatio-channel Attention Blocks for Cross-modal Crowd Counting](https://mlsanity.com/paper/2210.10392/) (2022)
- [RGB-T Multi-Modal Crowd Counting Based on Transformer](https://mlsanity.com/paper/2301.03033/) (2023)
- [Adaptive Generation of Privileged Intermediate Information for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2307.03240/) (2023)

## Built on by (most cited first, 11 of 11)

- [Multi-modal Crowd Counting via Modal Emulation](https://mlsanity.com/paper/2407.19491/) (2024)
- [FusionCounting: Robust visible-infrared image fusion guided by crowd counting via multi-task learning](https://mlsanity.com/paper/2508.20817/) (2025)
- [Depth-Assisted Network for Indiscernible Marine Object Counting with Adaptive Motion-Differentiated Feature Encoding](https://mlsanity.com/paper/2503.08152/) (2025)
- [Embodied Crowd Counting](https://mlsanity.com/paper/2503.08367/) (2025)
- [L2HCount:Generalizing Crowd Counting from Low to High Crowd Density via Density Simulation](https://mlsanity.com/paper/2503.12935/) (2025)
- [A Dual-Modulation Framework for RGB-T Crowd Counting via Spatially Modulated Attention and Adaptive Fusion](https://mlsanity.com/paper/2509.17079/) (2025)
- [Granular Ball Guided Stable Latent Domain Discovery for Domain-General Crowd Counting](https://mlsanity.com/paper/2603.24106/) (2026)
- [RACANet: Reliability-Aware Crowd Anchor Network for RGB-T Crowd Counting](https://mlsanity.com/paper/2604.24543/) (2026)
- [Thermal-Only Crowd Counting with Deployment-Time Privacy Protection](https://mlsanity.com/paper/2605.17042/) (2026)
- [A Benchmark for Semi-supervised Multi-modal Crowd Counting](https://mlsanity.com/paper/2606.03646/) (2026)
- [CrowdCue: Specialist-Cue Conditioning for Vision-Language Crowd Counting](https://mlsanity.com/paper/2609.23012/) (2026)

## Abstract (arXiv)

Multi-modal crowd counting involves estimating crowd density from both visual and thermal/depth images. This task is challenging due to the significant gap between these distinct modalities. In this paper, we propose a novel approach by introducing an auxiliary broker modality and on this basis frame the task as a triple-modal learning problem. We devise a fusion-based method to generate this broker modality, leveraging a non-diffusion, lightweight counterpart of modern denoising diffusion-based fusion models. Additionally, we identify and address the ghosting effect caused by direct cross-modal image fusion in multi-modal crowd counting. Through extensive experimental evaluations on popular multi-modal crowd-counting datasets, we demonstrate the effectiveness of our method, which introduces only 4 million additional parameters, yet achieves promising results. The code is available at https://github.com/HenryCilence/Broker-Modality-Crowd-Counting.