# Towards Universal Physical Attacks On Cascaded Camera-Lidar 3D Object Detection Models

arXiv: https://arxiv.org/abs/2101.10747 · HTML: https://mlsanity.com/paper/2101.10747/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 22 references that are in the map.
- First posted on arXiv 26 Jan 2021.

## Builds on (most cited first, 18 of 18)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Fooling automated surveillance cameras: adversarial patches to attack person detection](https://mlsanity.com/paper/1904.08653/) (2019)
- [EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection](https://mlsanity.com/paper/2007.08856/) (2020)
- [MVX-Net: Multimodal VoxelNet for 3D Object Detection](https://mlsanity.com/paper/1904.01649/) (2019)
- [Generating 3D Adversarial Point Clouds](https://mlsanity.com/paper/1809.07016/) (2018)
- [Physically Realizable Adversarial Examples for LiDAR Object Detection](https://mlsanity.com/paper/2004.00543/) (2020)
- [Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection](https://mlsanity.com/paper/1903.01864/) (2019)
- [Adversarial Attacks Beyond the Image Space](https://mlsanity.com/paper/1711.07183/) (2017)
- [Adversarial Attack and Defense on Point Sets](https://mlsanity.com/paper/1902.10899/) (2019)
- [Adversarial Objects Against LiDAR-Based Autonomous Driving Systems](https://mlsanity.com/paper/1907.05418/) (2019)
- [IPOD: Intensive Point-based Object Detector for Point Cloud](https://mlsanity.com/paper/1812.05276/) (2018)
- [RoarNet: A Robust 3D Object Detection based on RegiOn Approximation Refinement](https://mlsanity.com/paper/1811.03818/) (2018)
- [Soft Rasterizer: Differentiable Rendering for Unsupervised Single-View Mesh Reconstruction](https://mlsanity.com/paper/1901.05567/) (2019)

## Built on by (most cited first, 14 of 14)

- [Understanding the Robustness of 3D Object Detection with Bird's-Eye-View Representations in Autonomous Driving](https://mlsanity.com/paper/2303.17297/) (2023)
- [CausalAF: Causal Autoregressive Flow for Safety-Critical Driving Scenario Generation](https://mlsanity.com/paper/2110.13939/) (2021)
- [Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generation](https://mlsanity.com/paper/2404.08570/) (2024)
- [Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks](https://mlsanity.com/paper/2207.10170/) (2022)
- [Visual Adversarial Attacks and Defenses in the Physical World: A Survey](https://mlsanity.com/paper/2211.01671/) (2022)
- [Fusion is Not Enough: Single Modal Attacks on Fusion Models for 3D Object Detection](https://mlsanity.com/paper/2304.14614/) (2023)
- [Invariance-Aware Randomized Smoothing Certificates](https://mlsanity.com/paper/2211.14207/) (2022)
- [Benchmarking Robustness of AI-Enabled Multi-sensor Fusion Systems: Challenges and Opportunities](https://mlsanity.com/paper/2306.03454/) (2023)
- [Navigating Threats: A Survey of Physical Adversarial Attacks on LiDAR Perception Systems in Autonomous Vehicles](https://mlsanity.com/paper/2409.20426/) (2024)
- [RenderBender: A Survey on Adversarial Attacks Using Differentiable Rendering](https://mlsanity.com/paper/2411.09749/) (2024)
- [Semantically Adversarial Scenario Generation with Explicit Knowledge Guidance](https://mlsanity.com/paper/2106.04066/) (2021)
- [LiDAttack: Robust Black-box Attack on LiDAR-based Object Detection](https://mlsanity.com/paper/2411.01889/) (2024)
- [SABER: Spatially Consistent 3D Universal Adversarial Objects for BEV Detectors](https://mlsanity.com/paper/2505.22499/) (2025)
- [Finding Needles in Haystack: Formal Generative Models for Efficient Massive Parallel Simulations](https://mlsanity.com/paper/2301.01594/) (2023)

## Abstract (arXiv)

We propose a universal and physically realizable adversarial attack on a cascaded multi-modal deep learning network (DNN), in the context of self-driving cars. DNNs have achieved high performance in 3D object detection, but they are known to be vulnerable to adversarial attacks. These attacks have been heavily investigated in the RGB image domain and more recently in the point cloud domain, but rarely in both domains simultaneously - a gap to be filled in this paper. We use a single 3D mesh and differentiable rendering to explore how perturbing the mesh's geometry and texture can reduce the robustness of DNNs to adversarial attacks. We attack a prominent cascaded multi-modal DNN, the Frustum-Pointnet model. Using the popular KITTI benchmark, we showed that the proposed universal multi-modal attack was successful in reducing the model's ability to detect a car by nearly 73%. This work can aid in the understanding of what the cascaded RGB-point cloud DNN learns and its vulnerability to adversarial attacks.