# 3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection

arXiv: https://arxiv.org/abs/2112.04764 · HTML: https://mlsanity.com/paper/2112.04764/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 45 references that are in the map.
- First posted on arXiv 9 Dec 2021.

## Builds on (most cited first, 30 of 36)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Adversarial Examples: Attacks and Defenses for Deep Learning](https://mlsanity.com/paper/1712.07107/) (2017)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [Generating Adversarial Examples with Adversarial Networks](https://mlsanity.com/paper/1801.02610/) (2018)
- [How Does Mixup Help With Robustness and Generalization?](https://mlsanity.com/paper/2010.04819/) (2020)
- [Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving](https://mlsanity.com/paper/1907.06826/) (2019)
- [Train in Germany, Test in The USA: Making 3D Object Detectors Generalize](https://mlsanity.com/paper/2005.08139/) (2020)
- [Generating 3D Adversarial Point Clouds](https://mlsanity.com/paper/1809.07016/) (2018)
- [Physically Realizable Adversarial Examples for LiDAR Object Detection](https://mlsanity.com/paper/2004.00543/) (2020)
- [Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks](https://mlsanity.com/paper/2106.09249/) (2021)
- [AdvPC: Transferable Adversarial Perturbations on 3D Point Clouds](https://mlsanity.com/paper/1912.00461/) (2019)
- [Adversarial Attack and Defense on Point Sets](https://mlsanity.com/paper/1902.10899/) (2019)
- [Adversarial Objects Against LiDAR-Based Autonomous Driving Systems](https://mlsanity.com/paper/1907.05418/) (2019)
- [Improved Mixed-Example Data Augmentation](https://mlsanity.com/paper/1805.11272/) (2018)
- [Towards Generalization Across Depth for Monocular 3D Object Detection](https://mlsanity.com/paper/1912.08035/) (2019)
- [ADef: an Iterative Algorithm to Construct Adversarial Deformations](https://mlsanity.com/paper/1804.07729/) (2018)
- [R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes](https://mlsanity.com/paper/2108.04814/) (2021)

## Built on by (most cited first, 30 of 43)

- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [3D Semantic Segmentation in the Wild: Learning Generalized Models for Adverse-Condition Point Clouds](https://mlsanity.com/paper/2304.00690/) (2023)
- [Anomaly Detection under Distribution Shift](https://mlsanity.com/paper/2303.13845/) (2023)
- [Revisiting Domain-Adaptive 3D Object Detection by Reliable, Diverse and Class-balanced Pseudo-Labeling](https://mlsanity.com/paper/2307.07944/) (2023)
- [Domain generalization of 3D semantic segmentation in autonomous driving](https://mlsanity.com/paper/2212.04245/) (2022)
- [DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization](https://mlsanity.com/paper/2308.02282/) (2023)
- [Advancements in Point Cloud Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2308.12113/) (2023)
- [MATE: Masked Autoencoders are Online 3D Test-Time Learners](https://mlsanity.com/paper/2211.11432/) (2022)
- [A Comprehensive Study of the Robustness for LiDAR-based 3D Object Detectors against Adversarial Attacks](https://mlsanity.com/paper/2212.10230/) (2022)
- [A Survey of Label-Efficient Deep Learning for 3D Point Clouds](https://mlsanity.com/paper/2305.19812/) (2023)
- [Instant Domain Augmentation for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2303.14378/) (2023)
- [MDT3D: Multi-Dataset Training for LiDAR 3D Object Detection Generalization](https://mlsanity.com/paper/2308.01000/) (2023)
- [BEV-DG: Cross-Modal Learning under Bird's-Eye View for Domain Generalization of 3D Semantic Segmentation](https://mlsanity.com/paper/2308.06530/) (2023)
- [Walking Your LiDOG: A Journey Through Multiple Domains for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2304.11705/) (2023)
- [DG-PIC: Domain Generalized Point-In-Context Learning for Point Cloud Understanding](https://mlsanity.com/paper/2407.08801/) (2024)
- [V2X-DGW: Domain Generalization for Multi-agent Perception under Adverse Weather Conditions](https://mlsanity.com/paper/2403.11371/) (2024)
- [CloudFixer: Test-Time Adaptation for 3D Point Clouds via Diffusion-Guided Geometric Transformation](https://mlsanity.com/paper/2407.16193/) (2024)
- [Deep Transfer Learning for Intelligent Vehicle Perception: a Survey](https://mlsanity.com/paper/2306.15110/) (2023)
- [An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains](https://mlsanity.com/paper/2402.17562/) (2024)
- [Segmenting Known Objects and Unseen Unknowns without Prior Knowledge](https://mlsanity.com/paper/2209.05407/) (2022)
- [GeoAdapt: Self-Supervised Test-Time Adaptation in LiDAR Place Recognition Using Geometric Priors](https://mlsanity.com/paper/2308.04638/) (2023)
- [Multimodal 3D Object Detection on Unseen Domains](https://mlsanity.com/paper/2404.11764/) (2024)
- [Rethinking LiDAR Domain Generalization: Single Source as Multiple Density Domains](https://mlsanity.com/paper/2312.12098/) (2023)
- [MOS: Model Synergy for Test-Time Adaptation on LiDAR-Based 3D Object Detection](https://mlsanity.com/paper/2406.14878/) (2024)
- [Domain Generalization of 3D Object Detection by Density-Resampling](https://mlsanity.com/paper/2311.10845/) (2023)
- [Context-Aware Data Augmentation for LIDAR 3D Object Detection](https://mlsanity.com/paper/2211.10850/) (2022)
- [ParisLuco3D: A high-quality target dataset for domain generalization of LiDAR perception](https://mlsanity.com/paper/2310.16542/) (2023)
- [TTT-KD: Test-Time Training for 3D Semantic Segmentation through Knowledge Distillation from Foundation Models](https://mlsanity.com/paper/2403.11691/) (2024)
- [Ctrl-Crash: Controllable Diffusion for Realistic Car Crashes](https://mlsanity.com/paper/2506.00227/) (2025)
- [SAILOR: Scaling Anchors via Insights into Latent Object Representation](https://mlsanity.com/paper/2210.07811/) (2022)

## Abstract (arXiv)

As 3D object detection on point clouds relies on the geometrical relationships between the points, non-standard object shapes can hinder a method's detection capability. However, in safety-critical settings, robustness to out-of-domain and long-tail samples is fundamental to circumvent dangerous issues, such as the misdetection of damaged or rare cars. In this work, we substantially improve the generalization of 3D object detectors to out-of-domain data by deforming point clouds during training. We achieve this with 3D-VField: a novel data augmentation method that plausibly deforms objects via vector fields learned in an adversarial fashion. Our approach constrains 3D points to slide along their sensor view rays while neither adding nor removing any of them. The obtained vectors are transferable, sample-independent and preserve shape and occlusions. Despite training only on a standard dataset, such as KITTI, augmenting with our vector fields significantly improves the generalization to differently shaped objects and scenes. Towards this end, we propose and share CrashD: a synthetic dataset of realistic damaged and rare cars, with a variety of crash scenarios. Extensive experiments on KITTI, Waymo, our CrashD and SUN RGB-D show the generalizability of our techniques to out-of-domain data, different models and sensors, namely LiDAR and ToF cameras, for both indoor and outdoor scenes. Our CrashD dataset is available at https://crashd-cars.github.io.