# Understanding the Robustness of 3D Object Detection with Bird's-Eye-View Representations in Autonomous Driving

arXiv: https://arxiv.org/abs/2303.17297 · HTML: https://mlsanity.com/paper/2303.17297/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 56 of its 66 references that are in the map.
- First posted on arXiv 30 Mar 2023.

## Builds on (most cited first, 30 of 56)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Adversarial Patch](https://mlsanity.com/paper/1712.09665/) (2017)
- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming](https://mlsanity.com/paper/1907.07484/) (2019)
- [Adversarial Examples for Semantic Segmentation and Object Detection](https://mlsanity.com/paper/1703.08603/) (2017)
- [Fooling automated surveillance cameras: adversarial patches to attack person detection](https://mlsanity.com/paper/1904.08653/) (2019)
- [Understanding Robustness of Transformers for Image Classification](https://mlsanity.com/paper/2103.14586/) (2021)

## Built on by (most cited first, 30 of 37)

- [Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving](https://mlsanity.com/paper/2405.17426/) (2024)
- [Robustness of SAM: Segment Anything Under Corruptions and Beyond](https://mlsanity.com/paper/2306.07713/) (2023)
- [Is Your HD Map Constructor Reliable under Sensor Corruptions?](https://mlsanity.com/paper/2406.12214/) (2024)
- [UniMix: Towards Domain Adaptive and Generalizable LiDAR Semantic Segmentation in Adverse Weather](https://mlsanity.com/paper/2404.05145/) (2024)
- [RS2G: Data-Driven Scene-Graph Extraction and Embedding for Robust Autonomous Perception and Scenario Understanding](https://mlsanity.com/paper/2304.08600/) (2023)
- [ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Driving](https://mlsanity.com/paper/2403.02877/) (2024)
- [DIFFender: Diffusion-Based Adversarial Defense against Patch Attacks](https://mlsanity.com/paper/2306.09124/) (2023)
- [MSC-Bench: Benchmarking and Analyzing Multi-Sensor Corruption for Driving Perception](https://mlsanity.com/paper/2501.01037/) (2025)
- [SafeMap: Robust HD Map Construction from Incomplete Observations](https://mlsanity.com/paper/2507.00861/) (2025)
- [SeaBird: Segmentation in Bird's View with Dice Loss Improves Monocular 3D Detection of Large Objects](https://mlsanity.com/paper/2403.20318/) (2024)
- [Fusion is Not Enough: Single Modal Attacks on Fusion Models for 3D Object Detection](https://mlsanity.com/paper/2304.14614/) (2023)
- [Adv3D: Generating 3D Adversarial Examples for 3D Object Detection in Driving Scenarios with NeRF](https://mlsanity.com/paper/2309.01351/) (2023)
- [An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains](https://mlsanity.com/paper/2402.17562/) (2024)
- [What Really Matters for Robust Multi-Sensor HD Map Construction?](https://mlsanity.com/paper/2507.01484/) (2025)
- [Stability Under Scrutiny: Benchmarking Representation Paradigms for Online HD Mapping](https://mlsanity.com/paper/2510.10660/) (2025)
- [Hyperdimensional Uncertainty Quantification for Multimodal Uncertainty Fusion in Autonomous Vehicles Perception](https://mlsanity.com/paper/2503.20011/) (2025)
- [CHARM3R: Towards Unseen Camera Height Robust Monocular 3D Detector](https://mlsanity.com/paper/2508.11185/) (2025)
- [ExtrinSplat: Decoupling Geometry and Semantics for Open-Vocabulary Understanding in 3D Gaussian Splatting](https://mlsanity.com/paper/2509.22225/) (2025)
- [Robust Multimodal 3D Object Detection via Modality-Agnostic Decoding and Proximity-based Modality Ensemble](https://mlsanity.com/paper/2407.19156/) (2024)
- [Improving Robustness of LiDAR-Camera Fusion Model against Weather Corruption from Fusion Strategy Perspective](https://mlsanity.com/paper/2402.02738/) (2024)
- [Embodied Active Defense: Leveraging Recurrent Feedback to Counter Adversarial Patches](https://mlsanity.com/paper/2404.00540/) (2024)
- [DynamicPAE: Generating Scene-Aware Physical Adversarial Examples in Real-Time](https://mlsanity.com/paper/2412.08053/) (2024)
- [Team Samsung-RAL: Technical Report for 2024 RoboDrive Challenge-Robust Map Segmentation Track](https://mlsanity.com/paper/2405.10567/) (2024)
- [Towards Dependable Optical Flow: Benchmarks for Real-World Spatiotemporal Corruptions](https://mlsanity.com/paper/2411.14865/) (2024)
- [A Black-Box Evaluation Framework for Semantic Robustness in Bird's Eye View Detection](https://mlsanity.com/paper/2412.13913/) (2024)
- [Boosting Online 3D Multi-Object Tracking through Camera-Radar Cross Check](https://mlsanity.com/paper/2407.13937/) (2024)
- [An Investigation of Visual Foundation Models Robustness](https://mlsanity.com/paper/2508.16225/) (2025)
- [FlexMap: Robust HD Map Construction under Flexible Camera Configurations](https://mlsanity.com/paper/2601.22376/) (2026)
- [Improving Batch Normalization with TTA for Robust Object Detection in Self-Driving](https://mlsanity.com/paper/2411.18860/) (2024)
- [Towards Benchmarking and Assessing the Safety and Robustness of Autonomous Driving on Safety-critical Scenarios](https://mlsanity.com/paper/2503.23708/) (2025)

## Abstract (arXiv)

3D object detection is an essential perception task in autonomous driving to understand the environments. The Bird's-Eye-View (BEV) representations have significantly improved the performance of 3D detectors with camera inputs on popular benchmarks. However, there still lacks a systematic understanding of the robustness of these vision-dependent BEV models, which is closely related to the safety of autonomous driving systems. In this paper, we evaluate the natural and adversarial robustness of various representative models under extensive settings, to fully understand their behaviors influenced by explicit BEV features compared with those without BEV. In addition to the classic settings, we propose a 3D consistent patch attack by applying adversarial patches in the 3D space to guarantee the spatiotemporal consistency, which is more realistic for the scenario of autonomous driving. With substantial experiments, we draw several findings: 1) BEV models tend to be more stable than previous methods under different natural conditions and common corruptions due to the expressive spatial representations; 2) BEV models are more vulnerable to adversarial noises, mainly caused by the redundant BEV features; 3) Camera-LiDAR fusion models have superior performance under different settings with multi-modal inputs, but BEV fusion model is still vulnerable to adversarial noises of both point cloud and image. These findings alert the safety issue in the applications of BEV detectors and could facilitate the development of more robust models.