# PCGen: Point Cloud Generator for LiDAR Simulation

arXiv: https://arxiv.org/abs/2210.08738 · HTML: https://mlsanity.com/paper/2210.08738/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 17 Oct 2022.

## Builds on (most cited first, 20 of 20)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud](https://mlsanity.com/paper/1710.07368/) (2017)
- [SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud](https://mlsanity.com/paper/1809.08495/) (2018)
- [One Million Scenes for Autonomous Driving: ONCE Dataset](https://mlsanity.com/paper/2106.11037/) (2021)
- [Correlation Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1612.01939/) (2016)
- [LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World](https://mlsanity.com/paper/2006.09348/) (2020)
- [The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes](https://mlsanity.com/paper/1903.01568/) (2019)
- [Minimal-Entropy Correlation Alignment for Unsupervised Deep Domain Adaptation](https://mlsanity.com/paper/1711.10288/) (2017)
- [A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving](https://mlsanity.com/paper/1804.00103/) (2018)
- [Augmented LiDAR Simulator for Autonomous Driving](https://mlsanity.com/paper/1811.07112/) (2018)
- [Precise Synthetic Image and LiDAR (PreSIL) Dataset for Autonomous Vehicle Perception](https://mlsanity.com/paper/1905.00160/) (2019)
- [BLVD: Building A Large-scale 5D Semantics Benchmark for Autonomous Driving](https://mlsanity.com/paper/1903.06405/) (2019)

## Built on by (most cited first, 20 of 20)

- [LiDAR-NeRF: Novel LiDAR View Synthesis via Neural Radiance Fields](https://mlsanity.com/paper/2304.10406/) (2023)
- [RangeLDM: Fast Realistic LiDAR Point Cloud Generation](https://mlsanity.com/paper/2403.10094/) (2024)
- [LiDAR4D: Dynamic Neural Fields for Novel Space-time View LiDAR Synthesis](https://mlsanity.com/paper/2404.02742/) (2024)
- [Dynamic LiDAR Re-simulation using Compositional Neural Fields](https://mlsanity.com/paper/2312.05247/) (2023)
- [LiDAR-GS:Real-time LiDAR Re-Simulation using Gaussian Splatting](https://mlsanity.com/paper/2410.05111/) (2024)
- [LiDAR-CS Dataset: LiDAR Point Cloud Dataset with Cross-Sensors for 3D Object Detection](https://mlsanity.com/paper/2301.12515/) (2023)
- [MV-DeepSDF: Implicit Modeling with Multi-Sweep Point Clouds for 3D Vehicle Reconstruction in Autonomous Driving](https://mlsanity.com/paper/2309.16715/) (2023)
- [LiDAR-RT: Gaussian-based Ray Tracing for Dynamic LiDAR Re-simulation](https://mlsanity.com/paper/2412.15199/) (2024)
- [RealEngine: Simulating Autonomous Driving in Realistic Context](https://mlsanity.com/paper/2505.16902/) (2025)
- [GeoNLF: Geometry guided Pose-Free Neural LiDAR Fields](https://mlsanity.com/paper/2407.05597/) (2024)
- [InvRGB+L: Inverse Rendering of Complex Scenes with Unified Color and LiDAR Reflectance Modeling](https://mlsanity.com/paper/2507.17613/) (2025)
- [L-PR: Exploiting LiDAR Fiducial Marker for Unordered Low Overlap Multiview Point Cloud Registration](https://mlsanity.com/paper/2406.03298/) (2024)
- [SN-LiDAR: Semantic Neural Fields for Novel Space-time View LiDAR Synthesis](https://mlsanity.com/paper/2504.08361/) (2025)
- [Paved2Paradise: Cost-Effective and Scalable LiDAR Simulation by Factoring the Real World](https://mlsanity.com/paper/2312.01117/) (2023)
- [Simulation-Based Performance Evaluation of 3D Object Detection Methods with Deep Learning for a LiDAR Point Cloud Dataset in a SOTIF-related Use Case](https://mlsanity.com/paper/2503.03548/) (2025)
- [Physically Based Neural LiDAR Resimulation](https://mlsanity.com/paper/2507.12489/) (2025)
- [Comprehensive Assessment of LiDAR Evaluation Metrics: A Comparative Study Using Simulated and Real Data](https://mlsanity.com/paper/2511.02994/) (2025)
- [Neural-MMGS: Multi-modal Neural Gaussian Splats for Large-Scale Scene Reconstruction](https://mlsanity.com/paper/2509.17762/) (2025)
- [Physically-Based Simulation of Automotive LiDAR](https://mlsanity.com/paper/2512.05932/) (2025)
- [Spectral-Geometric Neural Fields for Pose-Free LiDAR View Synthesis](https://mlsanity.com/paper/2603.12903/) (2026)

## Abstract (arXiv)

Data is a fundamental building block for LiDAR perception systems. Unfortunately, real-world data collection and annotation is extremely costly & laborious. Recently, real data based LiDAR simulators have shown tremendous potential to complement real data, due to their scalability and high-fidelity compared to graphics engine based methods. Before simulation can be deployed in the real-world, two shortcomings need to be addressed. First, existing methods usually generate data which are more noisy and complete than the real point clouds, due to 3D reconstruction error and pure geometry-based raycasting method. Second, prior works on simulation for object detection focus solely on rigid objects, like cars, but VRUs, like pedestrians, are important road participants. To tackle the first challenge, we propose FPA raycasting and surrogate model raydrop. FPA enables the simulation of both point cloud coordinates and sensor features, while taking into account reconstruction noise. The ray-wise surrogate raydrop model mimics the physical properties of LiDAR's laser receiver to determine whether a simulated point would be recorded by a real LiDAR. With minimal training data, the surrogate model can generalize to different geographies and scenes, closing the domain gap between raycasted and real point clouds. To tackle the simulation of deformable VRU simulation, we employ SMPL dataset to provide a pedestrian simulation baseline and compare the domain gap between CAD and reconstructed objects. Applying our pipeline to perform novel sensor synthesis, results show that object detection models trained by simulation data can achieve similar result as the real data trained model.