# Learning to Drop Points for LiDAR Scan Synthesis

arXiv: https://arxiv.org/abs/2102.11952 · HTML: https://mlsanity.com/paper/2102.11952/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in World modeling from LiDAR generation, one of 11 directions in World models.
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Placed in

- World models / [World modeling from LiDAR generation](https://mlsanity.com/wm/world-modeling-from-lidar-generation/)

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [Differentiable Augmentation for Data-Efficient GAN Training](https://mlsanity.com/paper/2006.10738/) (2020)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation](https://mlsanity.com/paper/2003.14032/) (2020)
- [SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud](https://mlsanity.com/paper/1710.07368/) (2017)
- [LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World](https://mlsanity.com/paper/2006.09348/) (2020)
- [Vehicle Detection from 3D Lidar Using Fully Convolutional Network](https://mlsanity.com/paper/1608.07916/) (2016)
- [MisGAN: Learning from Incomplete Data with Generative Adversarial Networks](https://mlsanity.com/paper/1902.09599/) (2019)
- [Deep Generative Modeling of LiDAR Data](https://mlsanity.com/paper/1812.01180/) (2018)
- [Noise Robust Generative Adversarial Networks](https://mlsanity.com/paper/1911.11776/) (2019)

## Built on by (most cited first, 27 of 27)

- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [Learning to Generate Realistic LiDAR Point Clouds](https://mlsanity.com/paper/2209.03954/) (2022)
- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2309.09256/) (2023)
- [A Survey on Deep Domain Adaptation for LiDAR Perception](https://mlsanity.com/paper/2106.02377/) (2021)
- [DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes](https://mlsanity.com/paper/2410.18084/) (2024)
- [Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformer](https://mlsanity.com/paper/2407.19628/) (2024)
- [Is Your LiDAR Placement Optimized for 3D Scene Understanding?](https://mlsanity.com/paper/2403.17009/) (2024)
- [SoK: On the Semantic AI Security in Autonomous Driving](https://mlsanity.com/paper/2203.05314/) (2022)
- [Generative Range Imaging for Learning Scene Priors of 3D LiDAR Data](https://mlsanity.com/paper/2210.11750/) (2022)
- [Learning to Simulate Realistic LiDARs](https://mlsanity.com/paper/2209.10986/) (2022)
- [Fast LiDAR Data Generation with Rectified Flows](https://mlsanity.com/paper/2412.02241/) (2024)
- [WeatherGen: A Unified Diverse Weather Generator for LiDAR Point Clouds via Spider Mamba Diffusion](https://mlsanity.com/paper/2504.13561/) (2025)
- [La La LiDAR: Large-Scale Layout Generation from LiDAR Data](https://mlsanity.com/paper/2508.03691/) (2025)
- [Taming Transformers for Realistic Lidar Point Cloud Generation](https://mlsanity.com/paper/2404.05505/) (2024)
- [Fast LiDAR Upsampling using Conditional Diffusion Models](https://mlsanity.com/paper/2405.04889/) (2024)
- [L3DR: 3D-aware LiDAR Diffusion and Rectification](https://mlsanity.com/paper/2602.19064/) (2026)
- [SG-LDM: Semantic-Guided LiDAR Generation via Latent-Aligned Diffusion](https://mlsanity.com/paper/2506.23606/) (2025)
- [Differentiable SLAM Helps Deep Learning-based LiDAR Perception Tasks](https://mlsanity.com/paper/2309.09206/) (2023)
- [Paved2Paradise: Cost-Effective and Scalable LiDAR Simulation by Factoring the Real World](https://mlsanity.com/paper/2312.01117/) (2023)
- [Toward Physics-Aware Deep Learning Architectures for LiDAR Intensity Simulation](https://mlsanity.com/paper/2404.15774/) (2024)
- [GEM: Generating LiDAR World Model via Deformable Mamba](https://mlsanity.com/paper/2605.07326/) (2026)
- [A Unified Generative Framework for Realistic Lidar Simulation in Autonomous Driving Systems](https://mlsanity.com/paper/2312.15817/) (2023)
- [Data-driven Camera and Lidar Simulation Models for Autonomous Driving: A Review from Generative Models to Volume Renderers](https://mlsanity.com/paper/2402.10079/) (2024)
- [Anomalous Decision Discovery using Inverse Reinforcement Learning](https://mlsanity.com/paper/2507.04464/) (2025)
- [LiSTAR: Ray-Centric World Models for 4D LiDAR Sequences in Autonomous Driving](https://mlsanity.com/paper/2511.16049/) (2025)
- [DRUM: Diffusion-based Raydrop-aware Unpaired Mapping for Sim2Real LiDAR Segmentation](https://mlsanity.com/paper/2603.26263/) (2026)

## Abstract (arXiv)

3D laser scanning by LiDAR sensors plays an important role for mobile robots to understand their surroundings. Nevertheless, not all systems have high resolution and accuracy due to hardware limitations, weather conditions, and so on. Generative modeling of LiDAR data as scene priors is one of the promising solutions to compensate for unreliable or incomplete observations. In this paper, we propose a novel generative model for learning LiDAR data based on generative adversarial networks. As in the related studies, we process LiDAR data as a compact yet lossless representation, a cylindrical depth map. However, despite the smoothness of real-world objects, many points on the depth map are dropped out through the laser measurement, which causes learning difficulty on generative models. To circumvent this issue, we introduce measurement uncertainty into the generation process, which allows the model to learn a disentangled representation of the underlying shape and the dropout noises from a collection of real LiDAR data. To simulate the lossy measurement, we adopt a differentiable sampling framework to drop points based on the learned uncertainty. We demonstrate the effectiveness of our method on synthesis and reconstruction tasks using two datasets. We further showcase potential applications by restoring LiDAR data with various types of corruption.