# Efficient Masked AutoEncoder for Video Object Counting and A Large-Scale Benchmark

arXiv: https://arxiv.org/abs/2411.13056 · HTML: https://mlsanity.com/paper/2411.13056/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 20 Nov 2024.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [MultiMAE: Multi-modal Multi-task Masked Autoencoders](https://mlsanity.com/paper/2204.01678/) (2022)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [Bayesian Loss for Crowd Count Estimation with Point Supervision](https://mlsanity.com/paper/1908.03684/) (2019)
- [Context-Aware Crowd Counting](https://mlsanity.com/paper/1811.10452/) (2018)
- [CLIP-Count: Towards Text-Guided Zero-Shot Object Counting](https://mlsanity.com/paper/2305.07304/) (2023)
- [Boosting Crowd Counting via Multifaceted Attention](https://mlsanity.com/paper/2203.02636/) (2022)
- [TransCrowd: weakly-supervised crowd counting with transformers](https://mlsanity.com/paper/2104.09116/) (2021)
- [VLCounter: Text-aware Visual Representation for Zero-Shot Object Counting](https://mlsanity.com/paper/2312.16580/) (2023)
- [Point-Query Quadtree for Crowd Counting, Localization, and More](https://mlsanity.com/paper/2308.13814/) (2023)
- [Locality-constrained Spatial Transformer Network for Video Crowd Counting](https://mlsanity.com/paper/1907.07911/) (2019)
- [CCTrans: Simplifying and Improving Crowd Counting with Transformer](https://mlsanity.com/paper/2109.14483/) (2021)
- [Spatiotemporal Modeling for Crowd Counting in Videos](https://mlsanity.com/paper/1707.07890/) (2017)
- [Gramformer: Learning Crowd Counting via Graph-Modulated Transformer](https://mlsanity.com/paper/2401.03870/) (2024)
- [Redesigning Multi-Scale Neural Network for Crowd Counting](https://mlsanity.com/paper/2208.02894/) (2022)
- [Video Crowd Localization with Multi-focus Gaussian Neighborhood Attention and a Large-Scale Benchmark](https://mlsanity.com/paper/2107.08645/) (2021)
- [Estimating People Flows to Better Count Them in Crowded Scenes](https://mlsanity.com/paper/1911.10782/) (2019)
- [Fast Video Crowd Counting with a Temporal Aware Network](https://mlsanity.com/paper/1907.02198/) (2019)
- [Enhanced 3D convolutional networks for crowd counting](https://mlsanity.com/paper/1908.04121/) (2019)
- [A Spatio-Temporal Attentive Network for Video-Based Crowd Counting](https://mlsanity.com/paper/2208.11339/) (2022)
- [Motion-guided Non-local Spatial-Temporal Network for Video Crowd Counting](https://mlsanity.com/paper/2104.13946/) (2021)

## Built on by (most cited first, 1 of 1)

- [Object Counting Across Modalities: Taxonomies, Benchmarks, Applications, and Open Challenges](https://mlsanity.com/paper/2608.23845/) (2026)

## Abstract (arXiv)

The dynamic imbalance of the fore-background is a major challenge in video object counting, which is usually caused by the sparsity of target objects. This remains understudied in existing works and often leads to severe under-/over-prediction errors. To tackle this issue in video object counting, we propose a density-embedded Efficient Masked Autoencoder Counting (E-MAC) framework in this paper. To empower the model's representation ability on density regression, we develop a new mathttDensity-mathttEmbedded mathttMasked mmathttOdeling (mathttDEMO) method, which first takes the density map as an auxiliary modality to perform multimodal self-representation learning for image and density map. Although mathttDEMO contributes to effective cross-modal regression guidance, it also brings in redundant background information, making it difficult to focus on the foreground regions. To handle this dilemma, we propose an efficient spatial adaptive masking derived from density maps to boost efficiency. Meanwhile, we employ an optical flow-based temporal collaborative fusion strategy to effectively capture the dynamic variations across frames, aligning features to derive multi-frame density residuals. The counting accuracy of the current frame is boosted by harnessing the information from adjacent frames. In addition, considering that most existing datasets are limited to human-centric scenarios, we first propose a large video bird counting dataset, DroneBird, in natural scenarios for migratory bird protection. Extensive experiments on three crowd datasets and our textitDroneBird validate our superiority against the counterparts. The code and dataset are available.