# Learning Extremely High Density Crowds as Active Matters

arXiv: https://arxiv.org/abs/2503.12168 · HTML: https://mlsanity.com/paper/2503.12168/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 75 references that are in the map.
- First posted on arXiv 15 Mar 2025.

## Builds on (most cited first, 22 of 22)

- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [SimVP: Simpler yet Better Video Prediction](https://mlsanity.com/paper/2206.05099/) (2022)
- [PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2103.09504/) (2021)
- [PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning](https://mlsanity.com/paper/1804.06300/) (2018)
- [Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2206.12126/) (2022)
- [An End-to-End Transformer Model for Crowd Localization](https://mlsanity.com/paper/2202.13065/) (2022)
- [Localization in the Crowd with Topological Constraints](https://mlsanity.com/paper/2012.12482/) (2020)
- [Human Trajectory Prediction via Neural Social Physics](https://mlsanity.com/paper/2207.10435/) (2022)
- [Fast and Feature-Complete Differentiable Physics for Articulated Rigid Bodies with Contact](https://mlsanity.com/paper/2103.16021/) (2021)
- [Point-Query Quadtree for Crowd Counting, Localization, and More](https://mlsanity.com/paper/2308.13814/) (2023)
- [High-order Differentiable Autoencoder for Nonlinear Model Reduction](https://mlsanity.com/paper/2102.11026/) (2021)
- [MeshingNet: A New Mesh Generation Method based on Deep Learning](https://mlsanity.com/paper/2004.07016/) (2020)
- [Predicting Scene Parsing and Motion Dynamics in the Future](https://mlsanity.com/paper/1711.03270/) (2017)
- [CSCNet: Contextual Semantic Consistency Network for Trajectory Prediction in Crowded Spaces](https://mlsanity.com/paper/2202.08506/) (2022)
- [MMVP: Motion-Matrix-based Video Prediction](https://mlsanity.com/paper/2308.16154/) (2023)
- [Social Physics Informed Diffusion Model for Crowd Simulation](https://mlsanity.com/paper/2402.06680/) (2024)
- [Fine-grained differentiable physics: a yarn-level model for fabrics](https://mlsanity.com/paper/2202.00504/) (2022)
- [SocialCVAE: Predicting Pedestrian Trajectory via Interaction Conditioned Latents](https://mlsanity.com/paper/2402.17339/) (2024)
- [Informative Scene Decomposition for Crowd Analysis, Comparison and Simulation Guidance](https://mlsanity.com/paper/2004.14107/) (2020)
- [Consistency-Aware Anchor Pyramid Network for Crowd Localization](https://mlsanity.com/paper/2212.04067/) (2022)
- [Human Trajectory Forecasting with Explainable Behavioral Uncertainty](https://mlsanity.com/paper/2307.01817/) (2023)
- [Globally Continuous and Non-Markovian Activity Analysis from Videos](https://mlsanity.com/paper/1810.04954/) (2018)

## Built on by (most cited first, 1 of 1)

- [IndoorCrowd: A Multi-Scene Dataset for Human Detection, Segmentation, and Tracking with an Automated Annotation Pipeline](https://mlsanity.com/paper/2604.02032/) (2026)

## Abstract (arXiv)

Video-based high-density crowd analysis and prediction has been a long-standing topic in computer vision. It is notoriously difficult due to, but not limited to, the lack of high-quality data and complex crowd dynamics. Consequently, it has been relatively under studied. In this paper, we propose a new approach that aims to learn from in-the-wild videos, often with low quality where it is difficult to track individuals or count heads. The key novelty is a new physics prior to model crowd dynamics. We model high-density crowds as active matter, a continumm with active particles subject to stochastic forces, named 'crowd material'. Our physics model is combined with neural networks, resulting in a neural stochastic differential equation system which can mimic the complex crowd dynamics. Due to the lack of similar research, we adapt a range of existing methods which are close to ours for comparison. Through exhaustive evaluation, we show our model outperforms existing methods in analyzing and forecasting extremely high-density crowds. Furthermore, since our model is a continuous-time physics model, it can be used for simulation and analysis, providing strong interpretability. This is categorically different from most deep learning methods, which are discrete-time models and black-boxes.