mlsanityData to 14 Oct 2026

Learning to Generate Realistic LiDAR Point Clouds

arXiv 2209.03954 ↗First posted 8 Sep 2022cs.CVcited by 93

On the map: World models → World modeling from LiDAR generation

Abstract (arXiv)

We present LiDARGen, a novel, effective, and controllable generative model that produces realistic LiDAR point cloud sensory readings. Our method leverages the powerful score-matching energy-based model and formulates the point cloud generation process as a stochastic denoising process in the equirectangular view. This model allows us to sample diverse and high-quality point cloud samples with guaranteed physical feasibility and controllability. We validate the effectiveness of our method on the challenging KITTI-360 and NuScenes datasets. The quantitative and qualitative results show that our approach produces more realistic samples than other generative models. Furthermore, LiDARGen can sample point clouds conditioned on inputs without retraining. We demonstrate that our proposed generative model could be directly used to densify LiDAR point clouds. Our code is available at: https://www.zyrianov.org/lidargen/

Key facts. Cited by 93 papers in the map (data through 14 Oct 2026). Placed in World modeling from LiDAR generation, one of 11 directions in World models. Builds on 61 of its 83 references that are in the map. First posted on arXiv 8 Sep 2022.

Builds on

61 of 83 references.

Closest ancestors · 3

Same direction, newest first

  1. 2021Learning to Drop Points for LiDAR Scan Synthesis23 Feb
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space7 Jun
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation2 Dec

Other references · 58

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2013Auto-Encoding Variational Bayescited by 4,053
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  8. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  9. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  10. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  11. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  12. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
38 more
  1. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  2. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  3. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  4. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 735
  5. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
  6. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 657
  7. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  8. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 585
  9. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  10. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 565
  11. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  12. 2018Spherical CNNscited by 456
  13. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  14. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  15. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  16. 2017A Structured Self-attentive Sentence Embeddingcited by 318
  17. 2017Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphscited by 310
  18. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 307
  19. 2018SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filterscited by 261
  20. 2020Learning Gradient Fields for Shape Generationcited by 245
  21. 2016Volumetric and Multi-View CNNs for Object Classification on 3D Datacited by 240
  22. 2017SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloudcited by 232
  23. 2018PU-Net: Point Cloud Upsampling Networkcited by 209
  24. 2021Projected GANs Converge Fastercited by 208
  25. 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
  26. 2021Deep Parametric Continuous Convolutional Neural Networkscited by 161
  27. 2018SPLATNet: Sparse Lattice Networks for Point Cloud Processingcited by 158
  28. 2018OCNet: Object Context Network for Scene Parsingcited by 140
  29. 2020LiDARsim: Realistic LiDAR Simulation by Leveraging the Real Worldcited by 137
  30. 2016Vehicle Detection from 3D Lidar Using Fully Convolutional Networkcited by 107
  31. 2018PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attentioncited by 105
  32. 2018Multiresolution Tree Networks for 3D Point Cloud Processingcited by 91
  33. 2019Interpolated Convolutional Networks for 3D Point Cloud Understandingcited by 86
  34. 2018Local Spectral Graph Convolution for Point Set Feature Learningcited by 84
  35. 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
  36. 2018Point Cloud GANcited by 79
  37. 2018Deep Generative Modeling of LiDAR Datacited by 71
  38. 2019Multi-Angle Point Cloud-VAE: Unsupervised Feature Learning for 3D Point Clouds from Multiple Angles by Joint Self-Reconstruction and Half-to-Half Predictioncited by 66

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 13; 2024: 28; 2025: 34; 2026: 18.

93 papers.

Sort:

By area:

  1. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
  2. 2023UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generationcited by 78
  3. 2024Towards Realistic Scene Generation with LiDAR Diffusion Modelscited by 69
  4. 20253D and 4D World Modeling: A Surveycited by 67
  5. 2025A Survey of World Models for Autonomous Drivingcited by 60
  6. 2023LiDAR Data Synthesis with Denoising Diffusion Probabilistic Modelscited by 55
  7. 2024LidarDM: Generative LiDAR Simulation in a Generated Worldcited by 54
  8. 2024RangeLDM: Fast Realistic LiDAR Point Cloud Generationcited by 47
  9. 2023NeRF-LiDAR: Generating Realistic LiDAR Point Clouds with Neural Radiance Fieldscited by 44
  10. 2023Pyramid Diffusion for Fine 3D Large Scene Generationcited by 44
  11. 2024Scaling Diffusion Models to Real-World 3D LiDAR Scene Completioncited by 40
  12. 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenescited by 38
38 more
  1. 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
  2. 2025DRAWER: Digital Reconstruction and Articulation With Environment Realismcited by 29
  3. 2024Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformercited by 27
  4. 2024Is Your LiDAR Placement Optimized for 3D Scene Understanding?cited by 26
  5. 2024Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Videocited by 26
  6. 2023Dynamic LiDAR Re-simulation using Compositional Neural Fieldscited by 24
  7. 2024HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Drivingcited by 23
  8. 2025Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistencycited by 22
  9. 2023Look Ma, No Hands! Agent-Environment Factorization of Egocentric Videoscited by 21
  10. 2024ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretrainingcited by 21
  11. 2025X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllabilitycited by 20
  12. 2024Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passingcited by 19
  13. 2023Human-centric Scene Understanding for 3D Large-scale Scenarioscited by 17
  14. 2025LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequencescited by 17
  15. 2024DiffSSC: Semantic LiDAR Scan Completion using Denoising Diffusion Probabilistic Modelscited by 16
  16. 2024X-Drive: Cross-modality consistent multi-sensor data synthesis for driving scenarioscited by 14
  17. 2024Fast LiDAR Data Generation with Rectified Flowscited by 14
  18. 2025HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Videocited by 14
  19. 2025WeatherGen: A Unified Diverse Weather Generator for LiDAR Point Clouds via Spider Mamba Diffusioncited by 12
  20. 2024LiDAR-RT: Gaussian-based Ray Tracing for Dynamic LiDAR Re-simulationcited by 11
  21. 2023Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulationcited by 10
  22. 2025SPIRAL: Semantic-Aware Progressive LiDAR Scene Generation and Understandingcited by 10
  23. 2024LOGen: Toward Lidar Object Generation by Point Diffusioncited by 9
  24. 2026UniDriveDreamer: A Single-Stage Multimodal World Model for Autonomous Drivingcited by 9
  25. 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Methodcited by 8
  26. 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequencescited by 7
  27. 2025Transfer Your Perspective: Controllable 3D Generation from Any Viewpoint in a Driving Scenecited by 5
  28. 2025Towards Generating Realistic 3D Semantic Training Data for Autonomous Drivingcited by 5
  29. 2025Veila: Panoramic LiDAR Generation from a Monocular RGB Imagecited by 5
  30. 2025OmniGen: Unified Multimodal Sensor Generation for Autonomous Drivingcited by 5
  31. 2024Just Add 100 More: Augmenting NeRF-based Pseudo-LiDAR Point Cloud for Resolving Class-imbalance Problemcited by 4
  32. 2024Fast LiDAR Upsampling using Conditional Diffusion Modelscited by 4
  33. 2024LiDAR-EDIT: LiDAR Data Generation by Editing the Object Layouts in Real-World Scenescited by 4
  34. 2025MObI: Multimodal Object Inpainting Using Diffusion Modelscited by 4
  35. 2025A Self-Conditioned Representation Guided Diffusion Model for Realistic Text-to-LiDAR Scene Generationcited by 4
  36. 2024Generative AI Empowered LiDAR Point Cloud Generation with Multimodal Transformercited by 3
  37. 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
  38. 2026L3DR: 3D-aware LiDAR Diffusion and Rectificationcited by 3

50 most cited of 93 All 93 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026FPSGen: Flexible Point Cloud Scene Generation with BEV-Supported Transport Flows
  2. 2024Taming Transformers for Realistic Lidar Point Cloud Generation
  3. 2022Generative Range Imaging for Learning Scene Priors of 3D LiDAR Data
  4. 2025DriveLiDAR4D: Sequential and Controllable LiDAR Scene Generation for Autonomous Driving
  5. 2026HuLiGen: Human LiDAR Generation from Parametric Body Models
  6. 2025LiNeXt: Revisiting LiDAR Completion with Efficient Non-Diffusion Architectures
  7. 2025Re-Densification Meets Cross-Scale Propagation: Real-Time Neural Compression of LiDAR Point Clouds
  8. 2024Test-Time Adaptation of 3D Point Clouds via Denoising Diffusion Models