# DriveLiDAR4D: Sequential and Controllable LiDAR Scene Generation for Autonomous Driving

arXiv: https://arxiv.org/abs/2511.13309 · HTML: https://mlsanity.com/paper/2511.13309/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 21 of its 25 references that are in the map.
- First posted on arXiv 17 Nov 2025.

## Placed in

- End-to-end autonomous driving / [Generative simulation for driving](https://mlsanity.com/driving/generative-simulation-for-driving/)

## Builds on (most cited first, 21 of 21)

- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators](https://mlsanity.com/paper/2303.13439/) (2023)
- [Photorealistic Video Generation with Diffusion Models](https://mlsanity.com/paper/2312.06662/) (2023)
- [MagicDrive: Street View Generation with Diverse 3D Geometry Control](https://mlsanity.com/paper/2310.02601/) (2023)
- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation](https://mlsanity.com/paper/2307.01831/) (2023)
- [UniScene: Unified Occupancy-centric Driving Scene Generation](https://mlsanity.com/paper/2412.05435/) (2024)
- [UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation](https://mlsanity.com/paper/2311.01448/) (2023)
- [LidarDM: Generative LiDAR Simulation in a Generated World](https://mlsanity.com/paper/2404.02903/) (2024)
- [RangeLDM: Fast Realistic LiDAR Point Cloud Generation](https://mlsanity.com/paper/2403.10094/) (2024)
- [Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformer](https://mlsanity.com/paper/2407.19628/) (2024)
- [X-Drive: Cross-modality consistent multi-sensor data synthesis for driving scenarios](https://mlsanity.com/paper/2411.01123/) (2024)
- [OLiDM: Object-aware LiDAR Diffusion Models for Autonomous Driving](https://mlsanity.com/paper/2412.17226/) (2024)

## Built on by (most cited first, 2 of 2)

- [LaGen: Towards Autoregressive LiDAR Scene Generation](https://mlsanity.com/paper/2511.21256/) (2025)
- [LiFi: LiDAR Generation from Multi-View Images via Geometric and Semantic Collaborative Guidance](https://mlsanity.com/paper/2610.19786/) (2026)

## Abstract (arXiv)

The generation of realistic LiDAR point clouds plays a crucial role in the development and evaluation of autonomous driving systems. Although recent methods for 3D LiDAR point cloud generation have shown significant improvements, they still face notable limitations, including the lack of sequential generation capabilities and the inability to produce accurately positioned foreground objects and realistic backgrounds. These shortcomings hinder their practical applicability. In this paper, we introduce DriveLiDAR4D, a novel LiDAR generation pipeline consisting of multimodal conditions and a novel sequential noise prediction model LiDAR4DNet, capable of producing temporally consistent LiDAR scenes with highly controllable foreground objects and realistic backgrounds. To the best of our knowledge, this is the first work to address the sequential generation of LiDAR scenes with full scene manipulation capability in an end-to-end manner. We evaluated DriveLiDAR4D on the nuScenes and KITTI datasets, where we achieved an FRD score of 743.13 and an FVD score of 16.96 on the nuScenes dataset, surpassing the current state-of-the-art (SOTA) method, UniScene, with an performance boost of 37.2% in FRD and 24.1% in FVD, respectively.