mlsanityData to 14 Oct 2026

LidarDM: Generative LiDAR Simulation in a Generated World

arXiv 2404.02903 ↗First posted 3 Apr 2024cs.CV cs.ROcited by 54

Not on the map. Why?

Cited by 29 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present LidarDM, a novel LiDAR generative model capable of producing realistic, layout-aware, physically plausible, and temporally coherent LiDAR videos. LidarDM stands out with two unprecedented capabilities in LiDAR generative modeling: (i) LiDAR generation guided by driving scenarios, offering significant potential for autonomous driving simulations, and (ii) 4D LiDAR point cloud generation, enabling the creation of realistic and temporally coherent sequences. At the heart of our model is a novel integrated 4D world generation framework. Specifically, we employ latent diffusion models to generate the 3D scene, combine it with dynamic actors to form the underlying 4D world, and subsequently produce realistic sensory observations within this virtual environment. Our experiments indicate that our approach outperforms competing algorithms in realism, temporal coherency, and layout consistency. We additionally show that LidarDM can be used as a generative world model simulator for training and testing perception models.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 77 of its 86 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

77 of 86 references.

References · 77

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  9. 2017Neural Discrete Representation Learningcited by 4,712
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
38 more
  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  2. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  4. 2022Video Diffusion Modelscited by 2,305
  5. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  6. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  7. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  8. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  9. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  10. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  11. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  12. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  13. 2018Open3D: A Modern Library for 3D Data Processingcited by 721
  14. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 700
  15. 2016End-to-end Optimized Image Compressioncited by 696
  16. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  17. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  18. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  19. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  20. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  21. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  22. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
  23. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  24. 2023Learning Interactive Real-World Simulatorscited by 458
  25. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  26. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 346
  27. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 309
  28. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 297
  29. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  30. 2021End-to-end Interpretable Neural Motion Plannercited by 242
  31. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 242
  32. 2022Diffusion Probabilistic Modeling for Video Generationcited by 238
  33. 2021Projected GANs Converge Fastercited by 208
  34. 2022Robot Operating System 2: Design, Architecture, and Uses In The Wildcited by 199
  35. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
  36. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 177
  37. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 160
  38. 2020LiDARsim: Realistic LiDAR Simulation by Leveraging the Real Worldcited by 137

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 25; 2026: 19.

54 papers.

Sort:

By area:

  1. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  2. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
  3. 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
  4. 20253D and 4D World Modeling: A Surveycited by 67
  5. 2024SCube: Instant Large-Scale Scene Reconstruction using VoxSplatscited by 60
  6. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Modelscited by 56
  7. 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generationcited by 56
  8. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 51
  9. 2025DiST-4D: Disentangled Spatiotemporal Diffusion with Metric Depth for 4D Driving Scene Generationcited by 40
  10. 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Drivingcited by 37
  11. 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
  12. 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Drivingcited by 33
38 more
  1. 2025The Role of World Models in Shaping Autonomous Driving: A Comprehensive Surveycited by 29
  2. 2025DRAWER: Digital Reconstruction and Articulation With Environment Realismcited by 29
  3. 2025Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistencycited by 22
  4. 2025X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllabilitycited by 20
  5. 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Drivingcited by 20
  6. 2025MiLA: Multi-view Intensive-fidelity Long-term Video Generation World Model for Autonomous Drivingcited by 18
  7. 2024X-Drive: Cross-modality consistent multi-sensor data synthesis for driving scenarioscited by 14
  8. 2025COME: Adding Scene-Centric Forecasting Control to Occupancy World Modelcited by 14
  9. 2025HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Videocited by 14
  10. 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Drivingcited by 12
  11. 2024World Models: The Safety Perspectivecited by 9
  12. 2024LOGen: Toward Lidar Object Generation by Point Diffusioncited by 9
  13. 2025OccTENS: 3D Occupancy World Model via Temporal Next-Scale Predictioncited by 9
  14. 2026UniDriveDreamer: A Single-Stage Multimodal World Model for Autonomous Drivingcited by 9
  15. 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Methodcited by 8
  16. 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequencescited by 7
  17. 2025Transfer Your Perspective: Controllable 3D Generation from Any Viewpoint in a Driving Scenecited by 5
  18. 2025OmniGen: Unified Multimodal Sensor Generation for Autonomous Drivingcited by 5
  19. 2024LiDAR-EDIT: LiDAR Data Generation by Editing the Object Layouts in Real-World Scenescited by 4
  20. 2026L3DR: 3D-aware LiDAR Diffusion and Rectificationcited by 3
  21. 2026WorldFlow3D: Flowing Through 3D Distributions for Unbounded World Generationcited by 3
  22. 2026HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generationcited by 3
  23. 2026World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applicationscited by 3
  24. 2025JiSAM: Alleviate Labeling Burden and Corner Case Problems in Autonomous Driving via Minimal Real-World Datacited by 2
  25. 2025LiDAR-GS++:Improving LiDAR Gaussian Reconstruction via Diffusion Priorscited by 2
  26. 2025DriveLiDAR4D: Sequential and Controllable LiDAR Scene Generation for Autonomous Drivingcited by 2
  27. 2024Simultaneous Diffusion Sampling for Conditional LiDAR Generationcited by 1
  28. 2025LiDARDraft: Generating LiDAR Point Cloud from Versatile Inputscited by 1
  29. 2026Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perceptioncited by 1
  30. 2026Geometry Without Coordinates: LiDAR Diffusion as a 3D Feature Bridgecited by 1
  31. 2025RS2AD: End-to-End Autonomous Driving Data Generation from Roadside Sensor Observationscited by 0
  32. 2025LiSTAR: Ray-Centric World Models for 4D LiDAR Sequences in Autonomous Drivingcited by 0
  33. 2025Physically-Based Simulation of Automotive LiDARcited by 0
  34. 20254D-RaDiff: Latent Point Diffusion for 4D Radar Point Cloud Generationcited by 0
  35. 2026DRUM: Diffusion-based Raydrop-aware Unpaired Mapping for Sim2Real LiDAR Segmentationcited by 0
  36. 2026RS2AD-LiDAR: End-to-End Autonomous Driving LiDAR Data Generation from Roadside Sensor Observationscited by 0
  37. 2026PatchScene: Patch-based Voxel Diffusion for Large-Scale Scene Completioncited by 0
  38. 2026InfiniVerse: Occupancy Guided Unbounded Scene Generation for Autonomous Drivingcited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning to Generate 4D LiDAR Sequences
  2. 2025LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences
  3. 2025LaGen: Towards Autoregressive LiDAR Scene Generation
  4. 2026GEM: Generating LiDAR World Model via Deformable Mamba
  5. 2026HelloWorld: Towards Practical Applications of Generative Driving World Models
  6. 2026M4World: A Multi-view Multimodal Driving World Model for Interactive Object Manipulation and Minute-long Streaming
  7. 2026M3GD: Multi-Modal Multi-View Geometric Diffusion for Camera--LiDAR Novel View Synthesis
  8. 2025Range-Edit: Semantic Mask Guided Outdoor LiDAR Scene Editing