# How Do We Fail? Stress Testing Perception in Autonomous Vehicles

arXiv: https://arxiv.org/abs/2203.14155 · HTML: https://mlsanity.com/paper/2203.14155/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving.
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 26 Mar 2022.

## Placed in

- End-to-end autonomous driving / [Adversarial scenarios and safety testing](https://mlsanity.com/driving/adversarial-scenarios-and-safety-testing/)

## Builds on (most cited first, 24 of 24)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/1801.00553/) (2018)
- [Adversarial Examples: Attacks and Defenses for Deep Learning](https://mlsanity.com/paper/1712.07107/) (2017)
- [Adversarial Examples for Semantic Segmentation and Object Detection](https://mlsanity.com/paper/1703.08603/) (2017)
- [CoverNet: Multimodal Behavior Prediction using Trajectory Sets](https://mlsanity.com/paper/1911.10298/) (2019)
- [Physical Adversarial Examples for Object Detectors](https://mlsanity.com/paper/1807.07769/) (2018)
- [Generating 3D Adversarial Point Clouds](https://mlsanity.com/paper/1809.07016/) (2018)
- [PnPNet: End-to-End Perception and Prediction with Tracking in the Loop](https://mlsanity.com/paper/2005.14711/) (2020)
- [Transferable Adversarial Attacks for Image and Video Object Detection](https://mlsanity.com/paper/1811.12641/) (2018)
- [Lidar for Autonomous Driving: The principles, challenges, and trends for automotive lidar and perception systems](https://mlsanity.com/paper/2004.08467/) (2020)
- [Extending Adversarial Attacks and Defenses to Deep 3D Point Cloud Classifiers](https://mlsanity.com/paper/1901.03006/) (2019)
- [Adaptive Stress Testing for Autonomous Vehicles](https://mlsanity.com/paper/1902.01909/) (2019)
- [A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems](https://mlsanity.com/paper/2005.02979/) (2020)
- [Robustness of 3D Deep Learning in an Adversarial Setting](https://mlsanity.com/paper/1904.00923/) (2019)
- [Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection](https://mlsanity.com/paper/2107.07004/) (2021)
- [Out-of-Distribution Detection for Automotive Perception](https://mlsanity.com/paper/2011.01413/) (2020)
- [Deep Learning-based Vehicle Behaviour Prediction For Autonomous Driving Applications: A Review](https://mlsanity.com/paper/1912.11676/) (2019)
- [Robustness of Object Detectors in Degrading Weather Conditions](https://mlsanity.com/paper/2106.08795/) (2021)
- [Fooling LiDAR Perception via Adversarial Trajectory Perturbation](https://mlsanity.com/paper/2103.15326/) (2021)
- [Validation of Image-Based Neural Network Controllers through Adaptive Stress Testing](https://mlsanity.com/paper/2003.02381/) (2020)
- [Dynamic Environment Prediction in Urban Scenes using Recurrent Representation Learning](https://mlsanity.com/paper/1904.12374/) (2019)
- [Double-Prong ConvLSTM for Spatiotemporal Occupancy Prediction in Dynamic Environments](https://mlsanity.com/paper/2011.09045/) (2020)
- [Attention Augmented ConvLSTM for Environment Prediction](https://mlsanity.com/paper/2010.09662/) (2020)

## Built on by (most cited first, 18 of 18)

- [Survey on LiDAR Perception in Adverse Weather Conditions](https://mlsanity.com/paper/2304.06312/) (2023)
- [Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments](https://mlsanity.com/paper/2309.13893/) (2023)
- [The Role of Predictive Uncertainty and Diversity in Embodied AI and Robot Learning](https://mlsanity.com/paper/2405.03164/) (2024)
- [Failures Are Fated, But Can Be Faded: Characterizing and Mitigating Unwanted Behaviors in Large-Scale Vision and Language Models](https://mlsanity.com/paper/2406.07145/) (2024)
- [Testing Rare Downstream Safety Violations via Upstream Adaptive Sampling of Perception Error Models](https://mlsanity.com/paper/2209.09674/) (2022)
- [Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving](https://mlsanity.com/paper/2407.21126/) (2024)
- [RoboMD: Uncovering Robot Vulnerabilities through Semantic Potential Fields](https://mlsanity.com/paper/2412.02818/) (2024)
- [Monitoring of Perception Systems: Deterministic, Probabilistic, and Learning-based Fault Detection and Identification](https://mlsanity.com/paper/2205.10906/) (2022)
- [LOPR: Latent Occupancy PRediction using Generative Models](https://mlsanity.com/paper/2210.01249/) (2022)
- [Hard Cases Detection in Motion Prediction by Vision-Language Foundation Models](https://mlsanity.com/paper/2405.20991/) (2024)
- [Fairness in Autonomous Driving: Towards Understanding Confounding Factors in Object Detection under Challenging Weather](https://mlsanity.com/paper/2406.00219/) (2024)
- [Exploring Critical Testing Scenarios for Decision-Making Policies: An LLM Approach](https://mlsanity.com/paper/2412.06684/) (2024)
- [A novel framework for adaptive stress testing of autonomous vehicles in multi-lane roads](https://mlsanity.com/paper/2402.11813/) (2024)
- [ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events](https://mlsanity.com/paper/2308.14550/) (2023)
- [LLM-Assisted Red Teaming of Diffusion Models through "Failures Are Fated, But Can Be Faded"](https://mlsanity.com/paper/2410.16738/) (2024)
- [VLC Fusion: Vision-Language Conditioned Sensor Fusion for Robust Object Detection](https://mlsanity.com/paper/2505.12715/) (2025)
- [Scenario Generation for Testing of Autonomous Driving Systems Using Real-World Failure Records](https://mlsanity.com/paper/2606.31131/) (2026)
- [Beyond the Snowflakes: Disentangling Weather Effects from Confounding Meta-Factors in LiDAR Object Detection](https://mlsanity.com/paper/2610.13891/) (2026)

## Abstract (arXiv)

Autonomous vehicles (AVs) rely on environment perception and behavior prediction to reason about agents in their surroundings. These perception systems must be robust to adverse weather such as rain, fog, and snow. However, validation of these systems is challenging due to their complexity and dependence on observation histories. This paper presents a method for characterizing failures of LiDAR-based perception systems for AVs in adverse weather conditions. We develop a methodology based in reinforcement learning to find likely failures in object tracking and trajectory prediction due to sequences of disturbances. We apply disturbances using a physics-based data augmentation technique for simulating LiDAR point clouds in adverse weather conditions. Experiments performed across a wide range of driving scenarios from a real-world driving dataset show that our proposed approach finds high likelihood failures with smaller input disturbances compared to baselines while remaining computationally tractable. Identified failures can inform future development of robust perception systems for AVs.