# Learning to Generate 4D LiDAR Sequences

arXiv: https://arxiv.org/abs/2509.11959 · HTML: https://mlsanity.com/paper/2509.11959/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in World modeling from LiDAR generation, one of 11 directions in World models.
- Builds on 38 of its 39 references that are in the map.
- First posted on arXiv 15 Sep 2025.

## Placed in

- World models / [World modeling from LiDAR generation](https://mlsanity.com/wm/world-modeling-from-lidar-generation/)
- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 38)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [GAIA-1: A Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2309.17080/) (2023)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [MagicDrive: Street View Generation with Diverse 3D Geometry Control](https://mlsanity.com/paper/2310.02601/) (2023)
- [OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving](https://mlsanity.com/paper/2311.16038/) (2023)
- [DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation](https://mlsanity.com/paper/2403.06845/) (2024)
- [Robo3D: Towards Robust and Reliable 3D Perception against Corruptions](https://mlsanity.com/paper/2303.17597/) (2023)
- [Rethinking Range View Representation for LiDAR Segmentation](https://mlsanity.com/paper/2303.05367/) (2023)
- [Segment Any Point Cloud Sequences by Distilling Vision Foundation Models](https://mlsanity.com/paper/2306.09347/) (2023)
- [Epona: Autoregressive Diffusion World Model for Autonomous Driving](https://mlsanity.com/paper/2506.24113/) (2025)
- [UniScene: Unified Occupancy-centric Driving Scene Generation](https://mlsanity.com/paper/2412.05435/) (2024)
- [Street-View Image Generation from a Bird's-Eye View Layout](https://mlsanity.com/paper/2301.04634/) (2023)
- [DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Driving](https://mlsanity.com/paper/2408.00415/) (2024)
- [Learning to Generate Realistic LiDAR Point Clouds](https://mlsanity.com/paper/2209.03954/) (2022)
- [UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation](https://mlsanity.com/paper/2311.01448/) (2023)
- [OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Driving](https://mlsanity.com/paper/2405.20337/) (2024)
- [EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusion](https://mlsanity.com/paper/2405.00915/) (2024)
- [GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction](https://mlsanity.com/paper/2412.10373/) (2024)
- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2309.09256/) (2023)
- [RangeLDM: Fast Realistic LiDAR Point Cloud Generation](https://mlsanity.com/paper/2403.10094/) (2024)
- [DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes](https://mlsanity.com/paper/2410.18084/) (2024)
- [DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenes](https://mlsanity.com/paper/2409.04003/) (2024)
- [4D Contrastive Superflows are Dense 3D Representation Learners](https://mlsanity.com/paper/2407.06190/) (2024)
- [Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformer](https://mlsanity.com/paper/2407.19628/) (2024)
- [Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistency](https://mlsanity.com/paper/2506.07497/) (2025)
- [MMGDreamer: Mixed-Modality Graph for Geometry-Controllable 3D Indoor Scene Generation](https://mlsanity.com/paper/2502.05874/) (2025)

## Built on by (most cited first, 2 of 2)

- [MAD: Motion Appearance Decoupling for efficient Driving World Models](https://mlsanity.com/paper/2601.09452/) (2026)
- [GEM: Generating LiDAR World Model via Deformable Mamba](https://mlsanity.com/paper/2605.07326/) (2026)

## Abstract (arXiv)

While generative world models have advanced video and occupancy-based data synthesis, LiDAR generation remains underexplored despite its importance for accurate 3D perception. Extending generation to 4D LiDAR data introduces challenges in controllability, temporal stability, and evaluation. We present LiDARCrafter, a unified framework that converts free-form language into editable LiDAR sequences. Instructions are parsed into ego-centric scene graphs, which a tri-branch diffusion model transforms into object layouts, trajectories, and shapes. A range-image diffusion model generates the initial scan, and an autoregressive module extends it into a temporally coherent sequence. The explicit layout design further supports object-level editing, such as insertion or relocation. To enable fair assessment, we provide EvalSuite, a benchmark spanning scene-, object-, and sequence-level metrics. On nuScenes, LiDARCrafter achieves state-of-the-art fidelity, controllability, and temporal consistency, offering a foundation for LiDAR-based simulation and data augmentation.