# TransCrowd: weakly-supervised crowd counting with transformers

arXiv: https://arxiv.org/abs/2104.09116 · HTML: https://mlsanity.com/paper/2104.09116/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 87 references that are in the map.
- First posted on arXiv 19 Apr 2021.

## Builds on (most cited first, 30 of 53)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Visual Transformers: Token-based Image Representation and Processing for Computer Vision](https://mlsanity.com/paper/2006.03677/) (2020)
- [Medical Transformer: Gated Axial-Attention for Medical Image Segmentation](https://mlsanity.com/paper/2102.10662/) (2021)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds](https://mlsanity.com/paper/1808.01050/) (2018)
- [Bayesian Loss for Crowd Count Estimation with Point Supervision](https://mlsanity.com/paper/1908.03684/) (2019)
- [SceneFormer: Indoor Scene Generation with Transformers](https://mlsanity.com/paper/2012.09793/) (2020)
- [NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization](https://mlsanity.com/paper/2001.03360/) (2020)
- [Context-Aware Crowd Counting](https://mlsanity.com/paper/1811.10452/) (2018)
- [Learning from Synthetic Data for Crowd Counting in the Wild](https://mlsanity.com/paper/1903.03303/) (2019)
- [Distribution Matching for Crowd Counting](https://mlsanity.com/paper/2009.13077/) (2020)
- [End-to-end Lane Shape Prediction with Transformers](https://mlsanity.com/paper/2011.04233/) (2020)
- [JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method](https://mlsanity.com/paper/2004.03597/) (2020)
- [Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detection](https://mlsanity.com/paper/1906.07538/) (2019)
- [Localization in the Crowd with Topological Constraints](https://mlsanity.com/paper/2012.12482/) (2020)
- [Point in, Box out: Beyond Counting Persons in Crowds](https://mlsanity.com/paper/1904.01333/) (2019)

## Built on by (most cited first, 30 of 61)

- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [End-to-end Temporal Action Detection with Transformer](https://mlsanity.com/paper/2106.10271/) (2021)
- [UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation](https://mlsanity.com/paper/2406.01188/) (2024)
- [An End-to-End Transformer Model for Crowd Localization](https://mlsanity.com/paper/2202.13065/) (2022)
- [CrowdCLIP: Unsupervised Crowd Counting via Vision-Language Model](https://mlsanity.com/paper/2304.04231/) (2023)
- [Learning to Count Anything: Reference-less Class-agnostic Counting with Weak Supervision](https://mlsanity.com/paper/2205.10203/) (2022)
- [CCTrans: Simplifying and Improving Crowd Counting with Transformer](https://mlsanity.com/paper/2109.14483/) (2021)
- [Multi-Exit Vision Transformer for Dynamic Inference](https://mlsanity.com/paper/2106.15183/) (2021)
- [Vision Transformer Off-the-Shelf: A Surprising Baseline for Few-Shot Class-Agnostic Counting](https://mlsanity.com/paper/2305.04440/) (2023)
- [Learning Local and Global Temporal Contexts for Video Semantic Segmentation](https://mlsanity.com/paper/2204.03330/) (2022)
- [CrowdDiff: Multi-hypothesis Crowd Density Estimation using Diffusion Models](https://mlsanity.com/paper/2303.12790/) (2023)
- [Revisiting Crowd Counting: State-of-the-art, Trends, and Future Perspectives](https://mlsanity.com/paper/2209.07271/) (2022)
- [Degradation-Aware Self-Attention Based Transformer for Blind Image Super-Resolution](https://mlsanity.com/paper/2310.04180/) (2023)
- [CountFormer: Multi-View Crowd Counting Transformer](https://mlsanity.com/paper/2407.02047/) (2024)
- [Congested Crowd Instance Localization with Dilated Convolutional Swin Transformer](https://mlsanity.com/paper/2108.00584/) (2021)
- [Rethinking Global Context in Crowd Counting](https://mlsanity.com/paper/2105.10926/) (2021)
- [CrowdFormer: Weakly-supervised Crowd counting with Improved Generalizability](https://mlsanity.com/paper/2203.03768/) (2022)
- [CrowdMLP: Weakly-Supervised Crowd Counting via Multi-Granularity MLP](https://mlsanity.com/paper/2203.08219/) (2022)
- [Counting Varying Density Crowds Through Density Guided Adaptive Selection CNN and Transformer Estimation](https://mlsanity.com/paper/2206.10075/) (2022)
- [Focus for Free in Density-Based Counting](https://mlsanity.com/paper/2306.05129/) (2023)
- [Learning Discriminative Features for Crowd Counting](https://mlsanity.com/paper/2311.04509/) (2023)
- [RGB-T Multi-Modal Crowd Counting Based on Transformer](https://mlsanity.com/paper/2301.03033/) (2023)
- [Semi-supervised Counting via Pixel-by-pixel Density Distribution Modelling](https://mlsanity.com/paper/2402.15297/) (2024)
- [Boosting Few-shot Semantic Segmentation with Transformers](https://mlsanity.com/paper/2108.02266/) (2021)
- [Joint CNN and Transformer Network via weakly supervised Learning for efficient crowd counting](https://mlsanity.com/paper/2203.06388/) (2022)
- [Residual Swin Transformer Channel Attention Network for Image Demosaicing](https://mlsanity.com/paper/2204.07098/) (2022)
- [TreeFormer: a Semi-Supervised Transformer-based Framework for Tree Counting from a Single High Resolution Image](https://mlsanity.com/paper/2307.06118/) (2023)
- [Glance to Count: Learning to Rank with Anchors for Weakly-supervised Crowd Counting](https://mlsanity.com/paper/2205.14659/) (2022)
- [Exploring Unsupervised Cell Recognition with Prior Self-activation Maps](https://mlsanity.com/paper/2308.11144/) (2023)
- [FaceFormer: Scale-aware Blind Face Restoration with Transformers](https://mlsanity.com/paper/2207.09790/) (2022)

## Abstract (arXiv)

The mainstream crowd counting methods usually utilize the convolution neural network (CNN) to regress a density map, requiring point-level annotations. However, annotating each person with a point is an expensive and laborious process. During the testing phase, the point-level annotations are not considered to evaluate the counting accuracy, which means the point-level annotations are redundant. Hence, it is desirable to develop weakly-supervised counting methods that just rely on count-level annotations, a more economical way of labeling. Current weakly-supervised counting methods adopt the CNN to regress a total count of the crowd by an image-to-count paradigm. However, having limited receptive fields for context modeling is an intrinsic limitation of these weakly-supervised CNN-based methods. These methods thus cannot achieve satisfactory performance, with limited applications in the real world. The transformer is a popular sequence-to-sequence prediction model in natural language processing (NLP), which contains a global receptive field. In this paper, we propose TransCrowd, which reformulates the weakly-supervised crowd counting problem from the perspective of sequence-to-count based on transformers. We observe that the proposed TransCrowd can effectively extract the semantic crowd information by using the self-attention mechanism of transformer. To the best of our knowledge, this is the first work to adopt a pure transformer for crowd counting research. Experiments on five benchmark datasets demonstrate that the proposed TransCrowd achieves superior performance compared with all the weakly-supervised CNN-based counting methods and gains highly competitive counting performance compared with some popular fully-supervised counting methods.