# Consistent Valid Physically-Realizable Adversarial Attack against Crowd-flow Prediction Models

arXiv: https://arxiv.org/abs/2303.02669 · HTML: https://mlsanity.com/paper/2303.02669/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 75 references that are in the map.
- First posted on arXiv 5 Mar 2023.

## Builds on (most cited first, 30 of 31)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)
- [T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Prediction](https://mlsanity.com/paper/1811.05320/) (2018)
- [Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction](https://mlsanity.com/paper/1610.00081/) (2016)
- [MagNet: a Two-Pronged Defense against Adversarial Examples](https://mlsanity.com/paper/1705.09064/) (2017)
- [Stochastic Activation Pruning for Robust Adversarial Defense](https://mlsanity.com/paper/1803.01442/) (2018)
- [Do Wider Neural Networks Really Help Adversarial Robustness?](https://mlsanity.com/paper/2010.01279/) (2020)
- [Learning Traffic as Images: A Deep Convolutional Neural Network for Large-Scale Transportation Network Speed Prediction](https://mlsanity.com/paper/1701.04245/) (2017)
- [Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds](https://mlsanity.com/paper/2111.01395/) (2021)
- [Defensive Distillation is Not Robust to Adversarial Examples](https://mlsanity.com/paper/1607.04311/) (2016)
- [MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial Examples](https://mlsanity.com/paper/1711.08478/) (2017)
- [Harnessing Perceptual Adversarial Patches for Crowd Counting](https://mlsanity.com/paper/2109.07986/) (2021)
- [Are socially-aware trajectory prediction models really socially-aware?](https://mlsanity.com/paper/2108.10879/) (2021)
- [Practical Adversarial Attacks on Spatiotemporal Traffic Forecasting Models](https://mlsanity.com/paper/2210.02447/) (2022)
- [Harnessing the Vulnerability of Latent Layers in Adversarially Trained Models](https://mlsanity.com/paper/1905.05186/) (2019)
- [QuSecNets: Quantization-based Defense Mechanism for Securing Deep Neural Network against Adversarial Attacks](https://mlsanity.com/paper/1811.01437/) (2018)
- [PCNN: Deep Convolutional Networks for Short-term Traffic Congestion Prediction](https://mlsanity.com/paper/2003.07033/) (2020)
- [Adversarial Machine Learning And Speech Emotion Recognition: Utilizing Generative Adversarial Networks For Robustness](https://mlsanity.com/paper/1811.11402/) (2018)
- [Targeted Physical-World Attention Attack on Deep Learning Models in Road Sign Recognition](https://mlsanity.com/paper/2010.04331/) (2020)
- [Adversarial Attacks on Cognitive Self-Organizing Networks: The Challenge and the Way Forward](https://mlsanity.com/paper/1810.07242/) (2018)
- [SSCNets: Robustifying DNNs using Secure Selective Convolutional Filters](https://mlsanity.com/paper/1811.01443/) (2018)

## Built on by (most cited first, 1 of 1)

- [Membership Inference Attacks on DNNs using Adversarial Perturbations](https://mlsanity.com/paper/2307.05193/) (2023)

## Abstract (arXiv)

Recent works have shown that deep learning (DL) models can effectively learn city-wide crowd-flow patterns, which can be used for more effective urban planning and smart city management. However, DL models have been known to perform poorly on inconspicuous adversarial perturbations. Although many works have studied these adversarial perturbations in general, the adversarial vulnerabilities of deep crowd-flow prediction models in particular have remained largely unexplored. In this paper, we perform a rigorous analysis of the adversarial vulnerabilities of DL-based crowd-flow prediction models under multiple threat settings, making three-fold contributions. (1) We propose CaV-detect by formally identifying two novel properties - Consistency and Validity - of the crowd-flow prediction inputs that enable the detection of standard adversarial inputs with 0% false acceptance rate (FAR). (2) We leverage universal adversarial perturbations and an adaptive adversarial loss to present adaptive adversarial attacks to evade CaV-detect defense. (3) We propose CVPR, a Consistent, Valid and Physically-Realizable adversarial attack, that explicitly inducts the consistency and validity priors in the perturbation generation mechanism. We find out that although the crowd-flow models are vulnerable to adversarial perturbations, it is extremely challenging to simulate these perturbations in physical settings, notably when CaV-detect is in place. We also show that CVPR attack considerably outperforms the adaptively modified standard attacks in FAR and adversarial loss metrics. We conclude with useful insights emerging from our work and highlight promising future research directions.