# LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models

arXiv: https://arxiv.org/abs/2309.09256 · HTML: https://mlsanity.com/paper/2309.09256/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Placed in World modeling from LiDAR generation, one of 11 directions in World models.
- Builds on 33 of its 44 references that are in the map.
- First posted on arXiv 17 Sep 2023.

## Placed in

- World models / [World modeling from LiDAR generation](https://mlsanity.com/wm/world-modeling-from-lidar-generation/)
- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 33)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [Simple diffusion: End-to-end diffusion for high resolution images](https://mlsanity.com/paper/2301.11093/) (2023)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions](https://mlsanity.com/paper/1905.06292/) (2019)
- [Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review](https://mlsanity.com/paper/2005.09830/) (2020)
- [Learning to Generate Realistic LiDAR Point Clouds](https://mlsanity.com/paper/2209.03954/) (2022)
- [Monocular Depth Estimation using Diffusion Models](https://mlsanity.com/paper/2302.14816/) (2023)
- [Deep Generative Modeling of LiDAR Data](https://mlsanity.com/paper/1812.01180/) (2018)
- [Positional Encoding as Spatial Inductive Bias in GANs](https://mlsanity.com/paper/2012.05217/) (2020)
- [Scan-based Semantic Segmentation of LiDAR Point Clouds: An Experimental Study](https://mlsanity.com/paper/2004.11803/) (2020)
- [Learning to Drop Points for LiDAR Scan Synthesis](https://mlsanity.com/paper/2102.11952/) (2021)
- [Simulation-based Lidar Super-resolution for Ground Vehicles](https://mlsanity.com/paper/2004.05242/) (2020)
- [Generative Range Imaging for Learning Scene Priors of 3D LiDAR Data](https://mlsanity.com/paper/2210.11750/) (2022)

## Built on by (most cited first, 30 of 55)

- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [LidarDM: Generative LiDAR Simulation in a Generated World](https://mlsanity.com/paper/2404.02903/) (2024)
- [Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion](https://mlsanity.com/paper/2403.13470/) (2024)
- [DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes](https://mlsanity.com/paper/2410.18084/) (2024)
- [DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulation](https://mlsanity.com/paper/2411.11252/) (2024)
- [Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformer](https://mlsanity.com/paper/2407.19628/) (2024)
- [LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences](https://mlsanity.com/paper/2508.03692/) (2025)
- [DiffSSC: Semantic LiDAR Scan Completion using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2409.18092/) (2024)
- [Fast LiDAR Data Generation with Rectified Flows](https://mlsanity.com/paper/2412.02241/) (2024)
- [WeatherGen: A Unified Diverse Weather Generator for LiDAR Point Clouds via Spider Mamba Diffusion](https://mlsanity.com/paper/2504.13561/) (2025)
- [Distilling Diffusion Models to Efficient 3D LiDAR Scene Completion](https://mlsanity.com/paper/2412.03515/) (2024)
- [SPIRAL: Semantic-Aware Progressive LiDAR Scene Generation and Understanding](https://mlsanity.com/paper/2505.22643/) (2025)
- [LOGen: Toward Lidar Object Generation by Point Diffusion](https://mlsanity.com/paper/2412.07385/) (2024)
- [R2LDM: An Efficient 4D Radar Super-Resolution Framework Leveraging Diffusion Model](https://mlsanity.com/paper/2503.17097/) (2025)
- [U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequences](https://mlsanity.com/paper/2512.02982/) (2025)
- [DualDiff: Dual-branch Diffusion Model for Autonomous Driving with Semantic Fusion](https://mlsanity.com/paper/2505.01857/) (2025)
- [Taming Transformers for Realistic Lidar Point Cloud Generation](https://mlsanity.com/paper/2404.05505/) (2024)
- [Transfer Your Perspective: Controllable 3D Generation from Any Viewpoint in a Driving Scene](https://mlsanity.com/paper/2502.06682/) (2025)
- [Towards Generating Realistic 3D Semantic Training Data for Autonomous Driving](https://mlsanity.com/paper/2503.21449/) (2025)
- [Veila: Panoramic LiDAR Generation from a Monocular RGB Image](https://mlsanity.com/paper/2508.03690/) (2025)
- [OmniGen: Unified Multimodal Sensor Generation for Autonomous Driving](https://mlsanity.com/paper/2512.14225/) (2025)
- [Just Add 100 More: Augmenting NeRF-based Pseudo-LiDAR Point Cloud for Resolving Class-imbalance Problem](https://mlsanity.com/paper/2403.11573/) (2024)
- [Fast LiDAR Upsampling using Conditional Diffusion Models](https://mlsanity.com/paper/2405.04889/) (2024)
- [MObI: Multimodal Object Inpainting Using Diffusion Models](https://mlsanity.com/paper/2501.03173/) (2025)
- [Diffusion Distillation With Direct Preference Optimization For Efficient 3D LiDAR Scene Completion](https://mlsanity.com/paper/2504.11447/) (2025)
- [A Self-Conditioned Representation Guided Diffusion Model for Realistic Text-to-LiDAR Scene Generation](https://mlsanity.com/paper/2511.19004/) (2025)
- [Generative AI Empowered LiDAR Point Cloud Generation with Multimodal Transformer](https://mlsanity.com/paper/2406.18542/) (2024)
- [COLSON: Controllable Learning-Based Social Navigation via Diffusion-Based Reinforcement Learning](https://mlsanity.com/paper/2503.13934/) (2025)
- [L3DR: 3D-aware LiDAR Diffusion and Rectification](https://mlsanity.com/paper/2602.19064/) (2026)
- [SG-LDM: Semantic-Guided LiDAR Generation via Latent-Aligned Diffusion](https://mlsanity.com/paper/2506.23606/) (2025)

## Abstract (arXiv)

Generative modeling of 3D LiDAR data is an emerging task with promising applications for autonomous mobile robots, such as scalable simulation, scene manipulation, and sparse-to-dense completion of LiDAR point clouds. While existing approaches have demonstrated the feasibility of image-based LiDAR data generation using deep generative models, they still struggle with fidelity and training stability. In this work, we present R2DM, a novel generative model for LiDAR data that can generate diverse and high-fidelity 3D scene point clouds based on the image representation of range and reflectance intensity. Our method is built upon denoising diffusion probabilistic models (DDPMs), which have shown impressive results among generative model frameworks in recent years. To effectively train DDPMs in the LiDAR domain, we first conduct an in-depth analysis of data representation, loss functions, and spatial inductive biases. Leveraging our R2DM model, we also introduce a flexible LiDAR completion pipeline based on the powerful capabilities of DDPMs. We demonstrate that our method surpasses existing methods in generating tasks on the KITTI-360 and KITTI-Raw datasets, as well as in the completion task on the KITTI-360 dataset. Our project page can be found at https://kazuto1011.github.io/r2dm.