# Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting

arXiv: https://arxiv.org/abs/2012.04529 · HTML: https://mlsanity.com/paper/2012.04529/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 30 of 52)

- [CLIP-Count: Towards Text-Guided Zero-Shot Object Counting](https://mlsanity.com/paper/2305.07304/) (2023)
- [Few-shot Object Counting with Similarity-Aware Feature Enhancement](https://mlsanity.com/paper/2201.08959/) (2022)
- [C2Former: Calibrated and Complementary Transformer for RGB-Infrared Object Detection](https://mlsanity.com/paper/2306.16175/) (2023)
- [Rethinking Spatial Invariance of Convolutional Networks for Object Counting](https://mlsanity.com/paper/2206.05253/) (2022)
- [Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection](https://mlsanity.com/paper/2209.13801/) (2022)
- [Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption](https://mlsanity.com/paper/2501.10761/) (2025)
- [Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks](https://mlsanity.com/paper/2303.15710/) (2023)
- [Visual Tuning](https://mlsanity.com/paper/2305.06061/) (2023)
- [ControlNet-XS: Rethinking the Control of Text-to-Image Diffusion Models as Feedback-Control Systems](https://mlsanity.com/paper/2312.06573/) (2023)
- [Glass Segmentation with RGB-Thermal Image Pairs](https://mlsanity.com/paper/2204.05453/) (2022)
- [GLF-CR: SAR-Enhanced Cloud Removal with Global-Local Fusion](https://mlsanity.com/paper/2206.02850/) (2022)
- [DR.VIC: Decomposition and Reasoning for Video Individual Counting](https://mlsanity.com/paper/2203.12335/) (2022)
- [Redesigning Multi-Scale Neural Network for Crowd Counting](https://mlsanity.com/paper/2208.02894/) (2022)
- [UniRGB-IR: A Unified Framework for Visible-Infrared Semantic Tasks via Adapter Tuning](https://mlsanity.com/paper/2404.17360/) (2024)
- [Revisiting Crowd Counting: State-of-the-art, Trends, and Future Perspectives](https://mlsanity.com/paper/2209.07271/) (2022)
- [TAFNet: A Three-Stream Adaptive Fusion Network for RGB-T Crowd Counting](https://mlsanity.com/paper/2202.08517/) (2022)
- [Spatio-channel Attention Blocks for Cross-modal Crowd Counting](https://mlsanity.com/paper/2210.10392/) (2022)
- [Multi-modal Crowd Counting via a Broker Modality](https://mlsanity.com/paper/2407.07518/) (2024)
- [Coarse to Fine: Domain Adaptive Crowd Counting via Adversarial Scoring Network](https://mlsanity.com/paper/2107.12858/) (2021)
- [Backdoor Attacks on Crowd Counting](https://mlsanity.com/paper/2207.05641/) (2022)
- [Forget Less, Count Better: A Domain-Incremental Self-Distillation Learning Benchmark for Lifelong Crowd Counting](https://mlsanity.com/paper/2205.03307/) (2022)
- [RGB-D Indiscernible Object Counting in Underwater Scenes](https://mlsanity.com/paper/2304.11677/) (2023)
- [M-SpecGene: Generalized Foundation Model for RGBT Multispectral Vision](https://mlsanity.com/paper/2507.16318/) (2025)
- [RGB-T Multi-Modal Crowd Counting Based on Transformer](https://mlsanity.com/paper/2301.03033/) (2023)
- [Aerial Images Meet Crowdsourced Trajectories: A New Approach to Robust Road Extraction](https://mlsanity.com/paper/2111.15119/) (2021)
- [Semi-Supervised Crowd Counting with Contextual Modeling: Facilitating Holistic Understanding of Crowd Scenes](https://mlsanity.com/paper/2310.10352/) (2023)
- [Unconstrained Face Sketch Synthesis via Perception-Adaptive Network and A New Benchmark](https://mlsanity.com/paper/2112.01019/) (2021)
- [SQLNet: Scale-Modulated Query and Localization Network for Few-Shot Class-Agnostic Counting](https://mlsanity.com/paper/2311.10011/) (2023)
- [ComPtr: Towards Diverse Bi-source Dense Prediction Tasks via A Simple yet General Complementary Transformer](https://mlsanity.com/paper/2307.12349/) (2023)
- [Multi-modal Crowd Counting via Modal Emulation](https://mlsanity.com/paper/2407.19491/) (2024)

## Abstract (arXiv)

Crowd counting is a fundamental yet challenging task, which desires rich information to generate pixel-wise crowd density maps. However, most previous methods only used the limited information of RGB images and cannot well discover potential pedestrians in unconstrained scenarios. In this work, we find that incorporating optical and thermal information can greatly help to recognize pedestrians. To promote future researches in this field, we introduce a large-scale RGBT Crowd Counting (RGBT-CC) benchmark, which contains 2,030 pairs of RGB-thermal images with 138,389 annotated people. Furthermore, to facilitate the multimodal crowd counting, we propose a cross-modal collaborative representation learning framework, which consists of multiple modality-specific branches, a modality-shared branch, and an Information Aggregation-Distribution Module (IADM) to capture the complementary information of different modalities fully. Specifically, our IADM incorporates two collaborative information transfers to dynamically enhance the modality-shared and modality-specific representations with a dual information propagation mechanism. Extensive experiments conducted on the RGBT-CC benchmark demonstrate the effectiveness of our framework for RGBT crowd counting. Moreover, the proposed approach is universal for multimodal crowd counting and is also capable to achieve superior performance on the ShanghaiTechRGBD dataset. Finally, our source code and benchmark are released at urlhttp://lingboliu.com/RGBTCrowdCounting.html.