mlsanityData to 7 Oct 2026

UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation

arXiv 2311.01448 ↗First posted 2 Nov 2023cs.CV cs.LG cs.ROcited by 76

Not on the map. Why?

Cited by 29 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

LiDAR provides accurate geometric measurements of the 3D world. Unfortunately, dense LiDARs are very expensive and the point clouds captured by low-beam LiDAR are often sparse. To address these issues, we present UltraLiDAR, a data-driven framework for scene-level LiDAR completion, LiDAR generation, and LiDAR manipulation. The crux of UltraLiDAR is a compact, discrete representation that encodes the point cloud's geometric structure, is robust to noise, and is easy to manipulate. We show that by aligning the representation of a sparse point cloud to that of a dense point cloud, we can densify the sparse point clouds as if they were captured by a real high-density LiDAR, drastically reducing the cost. Furthermore, by learning a prior over the discrete codebook, we can generate diverse, realistic LiDAR point clouds for self-driving. We evaluate the effectiveness of UltraLiDAR on sparse-to-dense LiDAR completion and LiDAR generation. Experiments show that densifying real-world point clouds with our approach can significantly improve the performance of downstream perception systems. Compared to prior art on LiDAR generation, our approach generates much more realistic point clouds. According to A/B test, over 98.5% of the time human participants prefer our results over those of previous methods.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Builds on 47 of its 61 references that are in the map. First posted on arXiv 2 Nov 2023.

Builds on

47 of 61 references.

References · 47

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2022Generative Adversarial Networkscited by 5,603
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  9. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  11. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  12. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
35 more
  1. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  2. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  3. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  4. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  5. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  6. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  7. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  8. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  9. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  10. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  11. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  12. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  13. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  14. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  15. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  16. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  17. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  18. 2017Sparsity Invariant CNNscited by 325
  19. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  20. 2021Projected GANs Converge Fastercited by 204
  21. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 187
  22. 2021PandaSet: Advanced Sensor Suite Dataset for Autonomous Drivingcited by 186
  23. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  24. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  25. 2020LiDARsim: Realistic LiDAR Simulation by Leveraging the Real Worldcited by 137
  26. 2020S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Cloudscited by 102
  27. 2021SPG: Unsupervised Domain Adaptation for 3D Object Detection via Semantic Point Generationcited by 93
  28. 2022Learning to Generate Realistic LiDAR Point Cloudscited by 91
  29. 2020House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generationcited by 86
  30. 2020Semantic Scene Completion using Local Deep Implicit Functions on LiDAR Datacited by 74
  31. 2018Deep Generative Modeling of LiDAR Datacited by 70
  32. 2017Improved Adversarial Systems for 3D Object Generation and Reconstructioncited by 66
  33. 2020Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Datacited by 30
  34. 2019LiDAR Sensor modeling and Data augmentation with GANs for Autonomous drivingcited by 22
  35. 2018Adversarial Semantic Scene Completion from a Single Depth Imagecited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 23; 2025: 34; 2026: 16.

76 papers.

Sort:

By area:

  1. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 102
  2. 2024Towards Realistic Scene Generation with LiDAR Diffusion Modelscited by 67
  3. 20253D and 4D World Modeling: A Surveycited by 64
  4. 2025A Survey of World Models for Autonomous Drivingcited by 60
  5. 2024LidarDM: Generative LiDAR Simulation in a Generated Worldcited by 53
  6. 2024RangeLDM: Fast Realistic LiDAR Point Cloud Generationcited by 45
  7. 2024Scaling Diffusion Models to Real-World 3D LiDAR Scene Completioncited by 40
  8. 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenescited by 37
  9. 2024G3R: Gradient Guided Generalizable Reconstructioncited by 29
  10. 2025DRAWER: Digital Reconstruction and Articulation With Environment Realismcited by 29
  11. 2024Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Videocited by 26
  12. 2024Is Your LiDAR Placement Optimized for 3D Scene Understanding?cited by 25
38 more
  1. 2024HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Drivingcited by 21
  2. 2023LightSim: Neural Lighting Simulation for Urban Scenescited by 20
  3. 2025X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllabilitycited by 20
  4. 2024Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passingcited by 17
  5. 2025LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequencescited by 14
  6. 2024Fast LiDAR Data Generation with Rectified Flowscited by 13
  7. 2024OLiDM: Object-aware LiDAR Diffusion Models for Autonomous Drivingcited by 13
  8. 2025HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Videocited by 13
  9. 2025Two by Two: Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulationcited by 12
  10. 2025WeatherGen: A Unified Diverse Weather Generator for LiDAR Point Clouds via Spider Mamba Diffusioncited by 12
  11. 2024Distilling Diffusion Models to Efficient 3D LiDAR Scene Completioncited by 10
  12. 2025LiDPM: Rethinking Point Diffusion for Lidar Scene Completioncited by 10
  13. 2025Towards foundational LiDAR world models with efficient latent flow matchingcited by 10
  14. 2025La La LiDAR: Large-Scale Layout Generation from LiDAR Datacited by 10
  15. 2024LOGen: Toward Lidar Object Generation by Point Diffusioncited by 9
  16. 2025Skip Mamba Diffusion for Monocular 3D Semantic Scene Completioncited by 9
  17. 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Methodcited by 8
  18. 2024Taming Transformers for Realistic Lidar Point Cloud Generationcited by 5
  19. 2025Transfer Your Perspective: Controllable 3D Generation from Any Viewpoint in a Driving Scenecited by 5
  20. 2025Veila: Panoramic LiDAR Generation from a Monocular RGB Imagecited by 5
  21. 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequencescited by 5
  22. 2025OmniGen: Unified Multimodal Sensor Generation for Autonomous Drivingcited by 5
  23. 2024LiDAR-EDIT: LiDAR Data Generation by Editing the Object Layouts in Real-World Scenescited by 4
  24. 2025MObI: Multimodal Object Inpainting Using Diffusion Modelscited by 4
  25. 2026GenAssets: Generating in-the-wild 3D Assets in Latent Spacecited by 4
  26. 2024NeuroGauss4D-PCI: 4D Neural Fields and Gaussian Deformation Fields for Point Cloud Interpolationcited by 3
  27. 2024Generative AI Empowered LiDAR Point Cloud Generation with Multimodal Transformercited by 3
  28. 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
  29. 2025CymbaDiff: Structured Spatial Diffusion for Sketch-based 3D Semantic Urban Scene Generationcited by 3
  30. 2023GLiDR: Topologically Regularized Graph Generative Network for Sparse LiDAR Point Cloudscited by 2
  31. 2024PillarGen: Enhancing Radar Point Cloud Density and Quality via Pillar-based Point Generation Networkcited by 2
  32. 2024Outdoor Scene Extrapolation with Hierarchical Generative Cellular Automatacited by 2
  33. 2025Diffusion-Based Generative Models for 3D Occupancy Prediction in Autonomous Drivingcited by 2
  34. 2025SG-LDM: Semantic-Guided LiDAR Generation via Latent-Aligned Diffusioncited by 2
  35. 2025Learning to Generate 4D LiDAR Sequencescited by 2
  36. 2025RadarGen: Automotive Radar Point Cloud Generation from Camerascited by 2
  37. 2026R3DPA: Leveraging 3D Representation Alignment and RGB Pretrained Priors for LiDAR Scene Generationcited by 2
  38. 2026LiFlow: Flow Matching for 3D LiDAR Scene Completioncited by 2

50 most cited of 76 All 76 (JSON)