# LiDAR-EDIT: LiDAR Data Generation by Editing the Object Layouts in Real-World Scenes

arXiv: https://arxiv.org/abs/2412.00592 · HTML: https://mlsanity.com/paper/2412.00592/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in World modeling from LiDAR generation, one of 11 directions in World models.
- Builds on 29 of its 32 references that are in the map.
- First posted on arXiv 30 Nov 2024.

## Placed in

- World models / [World modeling from LiDAR generation](https://mlsanity.com/wm/world-modeling-from-lidar-generation/)

## Builds on (most cited first, 29 of 29)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution](https://mlsanity.com/paper/2007.16100/) (2020)
- [UniSim: A Neural Closed-Loop Sensor Simulator](https://mlsanity.com/paper/2308.01898/) (2023)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking](https://mlsanity.com/paper/2303.11301/) (2023)
- [LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World](https://mlsanity.com/paper/2006.09348/) (2020)
- [Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion](https://mlsanity.com/paper/2311.01017/) (2023)
- [Learning to Generate Realistic LiDAR Point Clouds](https://mlsanity.com/paper/2209.03954/) (2022)
- [UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation](https://mlsanity.com/paper/2311.01448/) (2023)
- [Deep Generative Modeling of LiDAR Data](https://mlsanity.com/paper/1812.01180/) (2018)
- [Towards Realistic Scene Generation with LiDAR Diffusion Models](https://mlsanity.com/paper/2404.00815/) (2024)
- [Neural LiDAR Fields for Novel View Synthesis](https://mlsanity.com/paper/2305.01643/) (2023)
- [A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving](https://mlsanity.com/paper/1804.00103/) (2018)
- [LidarDM: Generative LiDAR Simulation in a Generated World](https://mlsanity.com/paper/2404.02903/) (2024)
- [LiDAR-NeRF: Novel LiDAR View Synthesis via Neural Radiance Fields](https://mlsanity.com/paper/2304.10406/) (2023)
- [NeRF-LiDAR: Generating Realistic LiDAR Point Clouds with Neural Radiance Fields](https://mlsanity.com/paper/2304.14811/) (2023)
- [Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion](https://mlsanity.com/paper/2403.13470/) (2024)
- [LiDAR4D: Dynamic Neural Fields for Novel Space-time View LiDAR Synthesis](https://mlsanity.com/paper/2404.02742/) (2024)
- [Dynamic LiDAR Re-simulation using Compositional Neural Fields](https://mlsanity.com/paper/2312.05247/) (2023)
- [AlignMiF: Geometry-Aligned Multimodal Implicit Field for LiDAR-Camera Joint Synthesis](https://mlsanity.com/paper/2402.17483/) (2024)
- [TULIP: Transformer for Upsampling of LiDAR Point Clouds](https://mlsanity.com/paper/2312.06733/) (2023)
- [GenMM: Geometrically and Temporally Consistent Multimodal Data Generation for Video and LiDAR](https://mlsanity.com/paper/2406.10722/) (2024)

## Built on by (most cited first, 4 of 4)

- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [Generative AI for Autonomous Driving: Frontiers and Opportunities](https://mlsanity.com/paper/2505.08854/) (2025)
- [Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges](https://mlsanity.com/paper/2603.09086/) (2026)
- [Range-Edit: Semantic Mask Guided Outdoor LiDAR Scene Editing](https://mlsanity.com/paper/2511.17269/) (2025)

## Abstract (arXiv)

We present LiDAR-EDIT, a novel paradigm for generating synthetic LiDAR data for autonomous driving. Our framework edits real-world LiDAR scans by introducing new object layouts while preserving the realism of the background environment. Compared to end-to-end frameworks that generate LiDAR point clouds from scratch, LiDAR-EDIT offers users full control over the object layout, including the number, type, and pose of objects, while keeping most of the original real-world background. Our method also provides object labels for the generated data. Compared to novel view synthesis techniques, our framework allows for the creation of counterfactual scenarios with object layouts significantly different from the original real-world scene. LiDAR-EDIT uses spherical voxelization to enforce correct LiDAR projective geometry in the generated point clouds by construction. During object removal and insertion, generative models are employed to fill the unseen background and object parts that were occluded in the original real LiDAR scans. Experimental results demonstrate that our framework produces realistic LiDAR scans with practical value for downstream tasks.