# RGB-T Multi-Modal Crowd Counting Based on Transformer

arXiv: https://arxiv.org/abs/2301.03033 · HTML: https://mlsanity.com/paper/2301.03033/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 8 Jan 2023.

## Builds on (most cited first, 30 of 30)

- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [Bayesian Loss for Crowd Count Estimation with Point Supervision](https://mlsanity.com/paper/1908.03684/) (2019)
- [Rethinking Counting and Localization in Crowds:A Purely Point-Based Framework](https://mlsanity.com/paper/2107.12746/) (2021)
- [Context-Aware Crowd Counting](https://mlsanity.com/paper/1811.10452/) (2018)
- [Distribution Matching for Crowd Counting](https://mlsanity.com/paper/2009.13077/) (2020)
- [End-to-end people detection in crowded scenes](https://mlsanity.com/paper/1506.04878/) (2015)
- [Switching Convolutional Neural Network for Crowd Counting](https://mlsanity.com/paper/1708.00199/) (2017)
- [Boosting Crowd Counting via Multifaceted Attention](https://mlsanity.com/paper/2203.02636/) (2022)
- [TransCrowd: weakly-supervised crowd counting with transformers](https://mlsanity.com/paper/2104.09116/) (2021)
- [Perspective-Guided Convolution Networks for Crowd Counting](https://mlsanity.com/paper/1909.06966/) (2019)
- [Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting](https://mlsanity.com/paper/2012.04529/) (2020)
- [Revisiting Perspective Information for Efficient Crowd Counting](https://mlsanity.com/paper/1807.01989/) (2018)
- [CNN-based Density Estimation and Crowd Counting: A Survey](https://mlsanity.com/paper/2003.12783/) (2020)
- [PCC Net: Perspective Crowd Counting via Spatial Convolutional Network](https://mlsanity.com/paper/1905.10085/) (2019)
- [CCTrans: Simplifying and Improving Crowd Counting with Transformer](https://mlsanity.com/paper/2109.14483/) (2021)
- [TAFNet: A Three-Stream Adaptive Fusion Network for RGB-T Crowd Counting](https://mlsanity.com/paper/2202.08517/) (2022)
- [Dense Scale Network for Crowd Counting](https://mlsanity.com/paper/1906.09707/) (2019)
- [Audio-Visual Transformer Based Crowd Counting](https://mlsanity.com/paper/2109.01926/) (2021)
- [Scene-Adaptive Attention Network for Crowd Counting](https://mlsanity.com/paper/2112.15509/) (2021)
- [Coarse- and Fine-grained Attention Network with Background-aware Loss for Crowd Density Map Estimation](https://mlsanity.com/paper/2011.03721/) (2020)
- [STNet: Scale Tree Network with Multi-level Auxiliator for Crowd Counting](https://mlsanity.com/paper/2012.10189/) (2020)
- [Congested Crowd Instance Localization with Dilated Convolutional Swin Transformer](https://mlsanity.com/paper/2108.00584/) (2021)
- [CrowdMLP: Weakly-Supervised Crowd Counting via Multi-Granularity MLP](https://mlsanity.com/paper/2203.08219/) (2022)
- [Joint CNN and Transformer Network via weakly supervised Learning for efficient crowd counting](https://mlsanity.com/paper/2203.06388/) (2022)
- [Hybrid attention network based on progressive embedding scale-context for crowd counting](https://mlsanity.com/paper/2106.02324/) (2021)
- [Reinforcing Local Feature Representation for Weakly-Supervised Dense Crowd Counting](https://mlsanity.com/paper/2202.10681/) (2022)
- [SSR-HEF: Crowd Counting with Multi-Scale Semantic Refining and Hard Example Focusing](https://mlsanity.com/paper/2204.07406/) (2022)

## Built on by (most cited first, 7 of 7)

- [Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption](https://mlsanity.com/paper/2501.10761/) (2025)
- [Multi-modal Crowd Counting via a Broker Modality](https://mlsanity.com/paper/2407.07518/) (2024)
- [ComPtr: Towards Diverse Bi-source Dense Prediction Tasks via A Simple yet General Complementary Transformer](https://mlsanity.com/paper/2307.12349/) (2023)
- [Multimodal Crowd Counting with Pix2Pix GANs](https://mlsanity.com/paper/2401.07591/) (2024)
- [A Dual-Modulation Framework for RGB-T Crowd Counting via Spatially Modulated Attention and Adaptive Fusion](https://mlsanity.com/paper/2509.17079/) (2025)
- [Thermal-Only Crowd Counting with Deployment-Time Privacy Protection](https://mlsanity.com/paper/2605.17042/) (2026)
- [A Benchmark for Semi-supervised Multi-modal Crowd Counting](https://mlsanity.com/paper/2606.03646/) (2026)

## Abstract (arXiv)

Crowd counting aims to estimate the number of persons in a scene. Most state-of-the-art crowd counting methods based on color images can't work well in poor illumination conditions due to invisible objects. With the widespread use of infrared cameras, crowd counting based on color and thermal images is studied. Existing methods only achieve multi-modal fusion without count objective constraint. To better excavate multi-modal information, we use count-guided multi-modal fusion and modal-guided count enhancement to achieve the impressive performance. The proposed count-guided multi-modal fusion module utilizes a multi-scale token transformer to interact two-modal information under the guidance of count information and perceive different scales from the token perspective. The proposed modal-guided count enhancement module employs multi-scale deformable transformer decoder structure to enhance one modality feature and count information by the other modality. Experiment in public RGBT-CC dataset shows that our method refreshes the state-of-the-art results. https://github.com/liuzywen/RGBTCC