# Joint CNN and Transformer Network via weakly supervised Learning for efficient crowd counting

arXiv: https://arxiv.org/abs/2203.06388 · HTML: https://mlsanity.com/paper/2203.06388/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 48 references that are in the map.
- First posted on arXiv 12 Mar 2022.

## Builds on (most cited first, 30 of 32)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [LocalViT: Analyzing Locality in Vision Transformers](https://mlsanity.com/paper/2104.05707/) (2021)
- [Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds](https://mlsanity.com/paper/1808.01050/) (2018)
- [Bayesian Loss for Crowd Count Estimation with Point Supervision](https://mlsanity.com/paper/1908.03684/) (2019)
- [Rethinking Counting and Localization in Crowds:A Purely Point-Based Framework](https://mlsanity.com/paper/2107.12746/) (2021)
- [NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization](https://mlsanity.com/paper/2001.03360/) (2020)
- [Context-Aware Crowd Counting](https://mlsanity.com/paper/1811.10452/) (2018)
- [Learning from Synthetic Data for Crowd Counting in the Wild](https://mlsanity.com/paper/1903.03303/) (2019)
- [Distribution Matching for Crowd Counting](https://mlsanity.com/paper/2009.13077/) (2020)
- [Switching Convolutional Neural Network for Crowd Counting](https://mlsanity.com/paper/1708.00199/) (2017)
- [TransCrowd: weakly-supervised crowd counting with transformers](https://mlsanity.com/paper/2104.09116/) (2021)
- [Crowd Counting and Density Estimation by Trellis Encoder-Decoder Network](https://mlsanity.com/paper/1903.00853/) (2019)
- [Perspective-Guided Convolution Networks for Crowd Counting](https://mlsanity.com/paper/1909.06966/) (2019)
- [CrowdNet: A Deep Convolutional Network for Dense Crowd Counting](https://mlsanity.com/paper/1608.06197/) (2016)
- [CNN-based Density Estimation and Crowd Counting: A Survey](https://mlsanity.com/paper/2003.12783/) (2020)
- [Towards Using Count-level Weak Supervision for Crowd Counting](https://mlsanity.com/paper/2003.00164/) (2020)
- [HA-CCN: Hierarchical Attention-based Crowd Counting Network](https://mlsanity.com/paper/1907.10255/) (2019)
- [CCTrans: Simplifying and Improving Crowd Counting with Transformer](https://mlsanity.com/paper/2109.14483/) (2021)
- [PaDNet: Pan-Density Crowd Counting](https://mlsanity.com/paper/1811.02805/) (2018)
- [Dual Path Multi-Scale Fusion Networks with Attention for Crowd Counting](https://mlsanity.com/paper/1902.01115/) (2019)

## Built on by (most cited first, 6 of 6)

- [Learning to Count Anything: Reference-less Class-agnostic Counting with Weak Supervision](https://mlsanity.com/paper/2205.10203/) (2022)
- [RGB-T Multi-Modal Crowd Counting Based on Transformer](https://mlsanity.com/paper/2301.03033/) (2023)
- [Counting Manatee Aggregations using Deep Neural Networks and Anisotropic Gaussian Kernel](https://mlsanity.com/paper/2311.02315/) (2023)
- [Vision Transformers for Weakly-Supervised Microorganism Enumeration](https://mlsanity.com/paper/2412.02250/) (2024)
- [MS-CLR: Multi-Skeleton Contrastive Learning for Human Action Recognition](https://mlsanity.com/paper/2508.14889/) (2025)
- [Count2Density: Crowd Density Estimation without Location-level Annotations](https://mlsanity.com/paper/2509.03170/) (2025)

## Abstract (arXiv)

Currently, for crowd counting, the fully supervised methods via density map estimation are the mainstream research directions. However, such methods need location-level annotation of persons in an image, which is time-consuming and laborious. Therefore, the weakly supervised method just relying upon the count-level annotation is urgently needed. Since CNN is not suitable for modeling the global context and the interactions between image patches, crowd counting with weakly supervised learning via CNN generally can not show good performance. The weakly supervised model via Transformer was sequentially proposed to model the global context and learn contrast features. However, the transformer directly partitions the crowd images into a series of tokens, which may not be a good choice due to each pedestrian being an independent individual, and the parameter number of the network is very large. Hence, we propose a Joint CNN and Transformer Network (JCTNet) via weakly supervised learning for crowd counting in this paper. JCTNet consists of three parts: CNN feature extraction module (CFM), Transformer feature extraction module (TFM), and counting regression module (CRM). In particular, the CFM extracts crowd semantic information features, then sends their patch partitions to TRM for modeling global context, and CRM is used to predict the number of people. Extensive experiments and visualizations demonstrate that JCTNet can effectively focus on the crowd regions and obtain superior weakly supervised counting performance on five mainstream datasets. The number of parameters of the model can be reduced by about 67%~73% compared with the pure Transformer works. We also tried to explain the phenomenon that a model constrained only by count-level annotations can still focus on the crowd regions. We believe our work can promote further research in this field.