# Exploring Data Augmentation for Multi-Modality 3D Object Detection

arXiv: https://arxiv.org/abs/2012.12741 · HTML: https://mlsanity.com/paper/2012.12741/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2020.

## Built on by (most cited first, 15 of 15)

- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [Multimodal Virtual Point 3D Detection](https://mlsanity.com/paper/2111.06881/) (2021)
- [Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion](https://mlsanity.com/paper/2203.09780/) (2022)
- [Multi-Modal 3D Object Detection in Autonomous Driving: a Survey](https://mlsanity.com/paper/2106.12735/) (2021)
- [EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2112.11088/) (2021)
- [Graph R-CNN: Towards Accurate 3D Object Detection with Semantic-Decorated Local Graph](https://mlsanity.com/paper/2208.03624/) (2022)
- [PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer](https://mlsanity.com/paper/2305.06621/) (2023)
- [DetMatch: Two Teachers are Better Than One for Joint 2D and 3D Semi-Supervised Object Detection](https://mlsanity.com/paper/2203.09510/) (2022)
- [Exploring Geometric Consistency for Monocular 3D Object Detection](https://mlsanity.com/paper/2104.05858/) (2021)
- [Lift3D: Synthesize 3D Training Data by Lifting 2D GAN to 3D Generative Radiance Field](https://mlsanity.com/paper/2304.03526/) (2023)
- [VIN: Voxel-based Implicit Network for Joint 3D Object Detection and Segmentation for Lidars](https://mlsanity.com/paper/2107.02980/) (2021)
- [Improving Distant 3D Object Detection Using 2D Box Supervision](https://mlsanity.com/paper/2403.09230/) (2024)
- [Exploring 2D Data Augmentation for 3D Monocular Object Detection](https://mlsanity.com/paper/2104.10786/) (2021)
- [ODM3D: Alleviating Foreground Sparsity for Semi-Supervised Monocular 3D Object Detection](https://mlsanity.com/paper/2310.18620/) (2023)
- [Reference-Guided Diffusion Inpainting For Multimodal Counterfactual Generation](https://mlsanity.com/paper/2507.23058/) (2025)

## Abstract (arXiv)

It is counter-intuitive that multi-modality methods based on point cloud and images perform only marginally better or sometimes worse than approaches that solely use point cloud. This paper investigates the reason behind this phenomenon. Due to the fact that multi-modality data augmentation must maintain consistency between point cloud and images, recent methods in this field typically use relatively insufficient data augmentation. This shortage makes their performance under expectation. Therefore, we contribute a pipeline, named transformation flow, to bridge the gap between single and multi-modality data augmentation with transformation reversing and replaying. In addition, considering occlusions, a point in different modalities may be occupied by different objects, making augmentations such as cut and paste non-trivial for multi-modality detection. We further present Multi-mOdality Cut and pAste (MoCa), which simultaneously considers occlusion and physical plausibility to maintain the multi-modality consistency. Without using ensemble of detectors, our multi-modality detector achieves new state-of-the-art performance on nuScenes dataset and competitive performance on KITTI 3D benchmark. Our method also wins the best PKL award in the 3rd nuScenes detection challenge. Code and models will be released at https://github.com/open-mmlab/mmdetection3d.