# Unsupervised Neural Sensor Models for Synthetic LiDAR Data Augmentation

arXiv: https://arxiv.org/abs/1911.10575 · HTML: https://mlsanity.com/paper/1911.10575/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2019.

## Built on by (most cited first, 8 of 8)

- [Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey](https://mlsanity.com/paper/2307.04370/) (2023)
- [A Survey on Deep Domain Adaptation for LiDAR Perception](https://mlsanity.com/paper/2106.02377/) (2021)
- [Learning to Simulate Realistic LiDARs](https://mlsanity.com/paper/2209.10986/) (2022)
- [Real3D-Aug: Point Cloud Augmentation by Placing Real Objects with Occlusion Handling for 3D Detection and Segmentation](https://mlsanity.com/paper/2206.07634/) (2022)
- [Domain Adaptation in LiDAR Semantic Segmentation via Alternating Skip Connections and Hybrid Learning](https://mlsanity.com/paper/2201.05585/) (2022)
- [A Unified Generative Framework for Realistic Lidar Simulation in Autonomous Driving Systems](https://mlsanity.com/paper/2312.15817/) (2023)
- [WHU-Synthetic: A Synthetic Perception Dataset for 3-D Multitask Model Research](https://mlsanity.com/paper/2402.19059/) (2024)
- [Exploring the Landscape for Generative Sequence Models for Specialized Data Synthesis](https://mlsanity.com/paper/2411.01929/) (2024)

## Abstract (arXiv)

Data scarcity is a bottleneck to machine learning-based perception modules, usually tackled by augmenting real data with synthetic data from simulators. Realistic models of the vehicle perception sensors are hard to formulate in closed form, and at the same time, they require the existence of paired data to be learned. In this work, we propose two unsupervised neural sensor models based on unpaired domain translations with CycleGANs and Neural Style Transfer techniques. We employ CARLA as the simulation environment to obtain simulated LiDAR point clouds, together with their annotations for data augmentation, and we use KITTI dataset as the real LiDAR dataset from which we learn the realistic sensor model mapping. Moreover, we provide a framework for data augmentation and evaluation of the developed sensor models, through extrinsic object detection task evaluation using YOLO network adapted to provide oriented bounding boxes for LiDAR Bird-eye-View projected point clouds. Evaluation is performed on unseen real LiDAR frames from KITTI dataset, with different amounts of simulated data augmentation using the two proposed approaches, showing improvement of 6% mAP for the object detection task, in favor of the augmenting LiDAR point clouds adapted with the proposed neural sensor models over the raw simulated LiDAR.