# A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving

arXiv: https://arxiv.org/abs/1804.00103 · HTML: https://mlsanity.com/paper/1804.00103/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2018.

## Built on by (most cited first, 30 of 54)

- [You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors](https://mlsanity.com/paper/2109.00182/) (2021)
- [STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset](https://mlsanity.com/paper/2203.09065/) (2022)
- [PointGuard: Provably Robust 3D Point Cloud Classification](https://mlsanity.com/paper/2103.03046/) (2021)
- [A Survey on Deep Domain Adaptation for LiDAR Perception](https://mlsanity.com/paper/2106.02377/) (2021)
- [LiDAR-NeRF: Novel LiDAR View Synthesis via Neural Radiance Fields](https://mlsanity.com/paper/2304.10406/) (2023)
- [NeRF-LiDAR: Generating Realistic LiDAR Point Clouds with Neural Radiance Fields](https://mlsanity.com/paper/2304.14811/) (2023)
- [Image2Point: 3D Point-Cloud Understanding with 2D Image Pretrained Models](https://mlsanity.com/paper/2106.04180/) (2021)
- [ConDA: Unsupervised Domain Adaptation for LiDAR Segmentation via Regularized Domain Concatenation](https://mlsanity.com/paper/2111.15242/) (2021)
- [Imperceptible Transfer Attack and Defense on 3D Point Cloud Classification](https://mlsanity.com/paper/2111.10990/) (2021)
- [GIPSO: Geometrically Informed Propagation for Online Adaptation in 3D LiDAR Segmentation](https://mlsanity.com/paper/2207.09763/) (2022)
- [Isometric 3D Adversarial Examples in the Physical World](https://mlsanity.com/paper/2210.15291/) (2022)
- [Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles](https://mlsanity.com/paper/2112.12252/) (2021)
- [PCGen: Point Cloud Generator for LiDAR Simulation](https://mlsanity.com/paper/2210.08738/) (2022)
- [No Time to Train: Empowering Non-Parametric Networks for Few-shot 3D Scene Segmentation](https://mlsanity.com/paper/2404.04050/) (2024)
- [Compositional 3D-aware Video Generation with LLM Director](https://mlsanity.com/paper/2409.00558/) (2024)
- [Federated Learning in Intelligent Transportation Systems: Recent Applications and Open Problems](https://mlsanity.com/paper/2309.11039/) (2023)
- [Few-shot Class-incremental Learning for 3D Point Cloud Objects](https://mlsanity.com/paper/2205.15225/) (2022)
- [LiDAR-CS Dataset: LiDAR Point Cloud Dataset with Cross-Sensors for 3D Object Detection](https://mlsanity.com/paper/2301.12515/) (2023)
- [KTNet: Knowledge Transfer for Unpaired 3D Shape Completion](https://mlsanity.com/paper/2111.11976/) (2021)
- [Neural Intrinsic Embedding for Non-rigid Point Cloud Matching](https://mlsanity.com/paper/2303.01038/) (2023)
- [3DHacker: Spectrum-based Decision Boundary Generation for Hard-label 3D Point Cloud Attack](https://mlsanity.com/paper/2308.07546/) (2023)
- [LOGen: Toward Lidar Object Generation by Point Diffusion](https://mlsanity.com/paper/2412.07385/) (2024)
- [Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluation](https://mlsanity.com/paper/2305.10361/) (2023)
- [Blind Quality Assessment of 3D Dense Point Clouds with Structure Guided Resampling](https://mlsanity.com/paper/2208.14603/) (2022)
- [Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analyzing Point Clouds at Target-Level](https://mlsanity.com/paper/2303.01899/) (2023)
- [Toward Availability Attacks in 3D Point Clouds](https://mlsanity.com/paper/2407.11011/) (2024)
- [Task-Aware Sampling Layer for Point-Wise Analysis](https://mlsanity.com/paper/2107.04291/) (2021)
- [Balanced Residual Distillation Learning for 3D Point Cloud Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2408.01356/) (2024)
- [Data generation using simulation technology to improve perception mechanism of autonomous vehicles](https://mlsanity.com/paper/2207.00191/) (2022)
- [Enhancing Diffusion-based Point Cloud Generation with Smoothness Constraint](https://mlsanity.com/paper/2404.02396/) (2024)

## Abstract (arXiv)

3D LiDAR scanners are playing an increasingly important role in autonomous driving as they can generate depth information of the environment. However, creating large 3D LiDAR point cloud datasets with point-level labels requires a significant amount of manual annotation. This jeopardizes the efficient development of supervised deep learning algorithms which are often data-hungry. We present a framework to rapidly create point clouds with accurate point-level labels from a computer game. The framework supports data collection from both auto-driving scenes and user-configured scenes. Point clouds from auto-driving scenes can be used as training data for deep learning algorithms, while point clouds from user-configured scenes can be used to systematically test the vulnerability of a neural network, and use the falsifying examples to make the neural network more robust through retraining. In addition, the scene images can be captured simultaneously in order for sensor fusion tasks, with a method proposed to do automatic calibration between the point clouds and captured scene images. We show a significant improvement in accuracy (+9%) in point cloud segmentation by augmenting the training dataset with the generated synthesized data. Our experiments also show by testing and retraining the network using point clouds from user-configured scenes, the weakness/blind spots of the neural network can be fixed.