# Fast LiDAR Upsampling using Conditional Diffusion Models

arXiv: https://arxiv.org/abs/2405.04889 · HTML: https://mlsanity.com/paper/2405.04889/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 16 of its 19 references that are in the map.
- First posted on arXiv 8 May 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Learning to Generate Realistic LiDAR Point Clouds](https://mlsanity.com/paper/2209.03954/) (2022)
- [Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentation](https://mlsanity.com/paper/2107.05399/) (2021)
- [Deep Generative Modeling of LiDAR Data](https://mlsanity.com/paper/1812.01180/) (2018)
- [LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2309.09256/) (2023)
- [Learning to Drop Points for LiDAR Scan Synthesis](https://mlsanity.com/paper/2102.11952/) (2021)
- [Simulation-based Lidar Super-resolution for Ground Vehicles](https://mlsanity.com/paper/2004.05242/) (2020)
- [Generative Range Imaging for Learning Scene Priors of 3D LiDAR Data](https://mlsanity.com/paper/2210.11750/) (2022)
- [Implicit LiDAR Network: LiDAR Super-Resolution via Interpolation Weight Prediction](https://mlsanity.com/paper/2203.06413/) (2022)

## Built on by (most cited first, 4 of 4)

- [Generative AI Agents in Autonomous Machines: A Safety Perspective](https://mlsanity.com/paper/2410.15489/) (2024)
- [Map Imagination Like Blind Humans: Group Diffusion Model for Robotic Map Generation](https://mlsanity.com/paper/2412.16908/) (2024)
- [Geometry Without Coordinates: LiDAR Diffusion as a 3D Feature Bridge](https://mlsanity.com/paper/2609.10322/) (2026)
- [Gait Sequence Upsampling using Diffusion Models for Single LiDAR Sensors](https://mlsanity.com/paper/2410.08680/) (2024)

## Abstract (arXiv)

The search for refining 3D LiDAR data has attracted growing interest motivated by recent techniques such as supervised learning or generative model-based methods. Existing approaches have shown the possibilities for using diffusion models to generate refined LiDAR data with high fidelity, although the performance and speed of such methods have been limited. These limitations make it difficult to execute in real-time, causing the approaches to struggle in real-world tasks such as autonomous navigation and human-robot interaction. In this work, we introduce a novel approach based on conditional diffusion models for fast and high-quality sparse-to-dense upsampling of 3D scene point clouds through an image representation. Our method employs denoising diffusion probabilistic models trained with conditional inpainting masks, which have been shown to give high performance on image completion tasks. We introduce a series of experiments, including multiple datasets, sampling steps, and conditional masks. This paper illustrates that our method outperforms the baselines in sampling speed and quality on upsampling tasks using the KITTI-360 dataset. Furthermore, we illustrate the generalization ability of our approach by simultaneously training on real-world and synthetic datasets, introducing variance in quality and environments.