# Augmented LiDAR Simulator for Autonomous Driving

arXiv: https://arxiv.org/abs/1811.07112 · HTML: https://mlsanity.com/paper/1811.07112/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2018.

## Built on by (most cited first, 30 of 41)

- [UniSim: A Neural Closed-Loop Sensor Simulator](https://mlsanity.com/paper/2308.01898/) (2023)
- [Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting](https://mlsanity.com/paper/2401.01339/) (2024)
- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey](https://mlsanity.com/paper/2307.04370/) (2023)
- [Learning to Generate Realistic LiDAR Point Clouds](https://mlsanity.com/paper/2209.03954/) (2022)
- [SceneGen: Learning to Generate Realistic Traffic Scenes](https://mlsanity.com/paper/2101.06541/) (2021)
- [GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving](https://mlsanity.com/paper/2101.06543/) (2021)
- [Neural LiDAR Fields for Novel View Synthesis](https://mlsanity.com/paper/2305.01643/) (2023)
- [LidarDM: Generative LiDAR Simulation in a Generated World](https://mlsanity.com/paper/2404.02903/) (2024)
- [Semi-supervised 3D Object Detection with Proficient Teachers](https://mlsanity.com/paper/2207.12655/) (2022)
- [NeRF-LiDAR: Generating Realistic LiDAR Point Clouds with Neural Radiance Fields](https://mlsanity.com/paper/2304.14811/) (2023)
- [Dynamic LiDAR Re-simulation using Compositional Neural Fields](https://mlsanity.com/paper/2312.05247/) (2023)
- [PCGen: Point Cloud Generator for LiDAR Simulation](https://mlsanity.com/paper/2210.08738/) (2022)
- [Enhancing Autonomous Vehicle Training with Language Model Integration and Critical Scenario Generation](https://mlsanity.com/paper/2404.08570/) (2024)
- [Learning to Simulate Realistic LiDARs](https://mlsanity.com/paper/2209.10986/) (2022)
- [Instant Domain Augmentation for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2303.14378/) (2023)
- [Pattern-Aware Data Augmentation for LiDAR 3D Object Detection](https://mlsanity.com/paper/2112.00050/) (2021)
- [LiDAR-CS Dataset: LiDAR Point Cloud Dataset with Cross-Sensors for 3D Object Detection](https://mlsanity.com/paper/2301.12515/) (2023)
- [CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splatting](https://mlsanity.com/paper/2507.18473/) (2025)
- [Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation](https://mlsanity.com/paper/2311.01446/) (2023)
- [OASim: an Open and Adaptive Simulator based on Neural Rendering for Autonomous Driving](https://mlsanity.com/paper/2402.03830/) (2024)
- [LOGen: Toward Lidar Object Generation by Point Diffusion](https://mlsanity.com/paper/2412.07385/) (2024)
- [TrainSim: A Railway Simulation Framework for LiDAR and Camera Dataset Generation](https://mlsanity.com/paper/2302.14486/) (2023)
- [Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analyzing Point Clouds at Target-Level](https://mlsanity.com/paper/2303.01899/) (2023)
- [Hardness-Aware Scene Synthesis for Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/2405.17422/) (2024)
- [Quasi-Balanced Self-Training on Noise-Aware Synthesis of Object Point Clouds for Closing Domain Gap](https://mlsanity.com/paper/2203.03833/) (2022)
- [Context-Aware Data Augmentation for LIDAR 3D Object Detection](https://mlsanity.com/paper/2211.10850/) (2022)
- [Weak-to-Strong 3D Object Detection with X-Ray Distillation](https://mlsanity.com/paper/2404.00679/) (2024)
- [Just Add 100 More: Augmenting NeRF-based Pseudo-LiDAR Point Cloud for Resolving Class-imbalance Problem](https://mlsanity.com/paper/2403.11573/) (2024)
- [ISETAuto: Detecting vehicles with depth and radiance information](https://mlsanity.com/paper/2101.01843/) (2021)

## Abstract (arXiv)

In Autonomous Driving (AD), detection and tracking of obstacles on the roads is a critical task. Deep-learning based methods using annotated LiDAR data have been the most widely adopted approach for this. Unfortunately, annotating 3D point cloud is a very challenging, time- and money-consuming task. In this paper, we propose a novel LiDAR simulator that augments real point cloud with synthetic obstacles (e.g., cars, pedestrians, and other movable objects). Unlike previous simulators that entirely rely on CG models and game engines, our augmented simulator bypasses the requirement to create high-fidelity background CAD models. Instead, we can simply deploy a vehicle with a LiDAR scanner to sweep the street of interests to obtain the background point cloud, based on which annotated point cloud can be automatically generated. This unique "scan-and-simulate" capability makes our approach scalable and practical, ready for large-scale industrial applications. In this paper, we describe our simulator in detail, in particular the placement of obstacles that is critical for performance enhancement. We show that detectors with our simulated LiDAR point cloud alone can perform comparably (within two percentage points) with these trained with real data. Mixing real and simulated data can achieve over 95% accuracy.