R3DPA: Leveraging 3D Representation Alignment and RGB Pretrained Priors for LiDAR Scene Generation
On the map: World models → World modeling from LiDAR generation
Abstract (arXiv)
LiDAR scene synthesis is an emerging solution to scarcity in 3D data for robotic tasks such as autonomous driving. Recent approaches employ diffusion or flow matching models to generate realistic scenes, but 3D data remains limited compared to RGB datasets with millions of samples. We introduce R3DPA, the first LiDAR scene generation method to unlock image-pretrained priors for LiDAR point clouds, and leverage self-supervised 3D representations for state-of-the-art results. Specifically, we (i) align intermediate features of our generative model with self-supervised 3D features, which substantially improves generation quality; (ii) transfer knowledge from large-scale image-pretrained generative models to LiDAR generation, mitigating limited LiDAR datasets; and (iii) enable point cloud control at inference for object inpainting and scene mixing with solely an unconditional model. On the KITTI-360 benchmark R3DPA achieves state of the art performance. Code and pretrained models are available at https://github.com/valeoai/R3DPA.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in World modeling from LiDAR generation, one of 11 directions in World models. Builds on 44 of its 51 references that are in the map. First posted on arXiv 12 Jan 2026.
Builds on
44 of 51 references.
Closest ancestors · 10
Same direction, newest first
- 2025SPIRAL: Semantic-Aware Progressive LiDAR Scene Generation and Understanding28 May
- 2025LiDPM: Rethinking Point Diffusion for Lidar Scene Completion24 Apr
- 2024OLiDM: Object-aware LiDAR Diffusion Models for Autonomous Driving23 Dec
- 2024LOGen: Toward Lidar Object Generation by Point Diffusion10 Dec
- 2024Fast LiDAR Data Generation with Rectified Flows3 Dec
- 2024Towards Realistic Scene Generation with LiDAR Diffusion Models31 Mar
- 2024Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion20 Mar
- 2024RangeLDM: Fast Realistic LiDAR Point Cloud Generation15 Mar
- 2023LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models17 Sep
- 2022Learning to Generate Realistic LiDAR Point Clouds8 Sep
Other references · 34
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022Generative Adversarial Networkscited by 5,603
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
22 more
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
- 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
- 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
- 2016Conditional Image Generation with PixelCNN Decoderscited by 859
- 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 707
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
- 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
- 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
- 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
- 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 376
- 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212
- 2025Sonata: Self-Supervised Learning of Reliable Point Representationscited by 108
- 2023RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Drivingcited by 76
- 2023UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generationcited by 76
- 2018A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Drivingcited by 54
- 2023Three Pillars improving Vision Foundation Model Distillation for Lidarcited by 43
- 2018Augmented LiDAR Simulator for Autonomous Drivingcited by 41
- 2020Scan-based Semantic Segmentation of LiDAR Point Clouds: An Experimental Studycited by 33
- 2025REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Trainingcited by 33
- 2025U-REPA: Aligning Diffusion U-Nets to ViTscited by 29
- 2023LIP-Loc: LiDAR Image Pretraining for Cross-Modal Localizationcited by 26
- 2025MObI: Multimodal Object Inpainting Using Diffusion Modelscited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026M3GD: Multi-Modal Multi-View Geometric Diffusion for Camera--LiDAR Novel View Synthesis
- 2026Points-to-3D: Structure-Aware 3D Generation with Point Cloud Priors
- 2025Learning to Generate 4D LiDAR Sequences
- 2025Veila: Panoramic LiDAR Generation from a Monocular RGB Image
- 2024Simultaneous Diffusion Sampling for Conditional LiDAR Generation
- 2022Generative Range Imaging for Learning Scene Priors of 3D LiDAR Data
- 2024GenMM: Geometrically and Temporally Consistent Multimodal Data Generation for Video and LiDAR
- 2025DriveLiDAR4D: Sequential and Controllable LiDAR Scene Generation for Autonomous Driving