# Generating Photo-realistic Images from LiDAR Point Clouds with Generative Adversarial Networks

arXiv: https://arxiv.org/abs/2112.11245 · HTML: https://mlsanity.com/paper/2112.11245/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 20 Dec 2021.

## Builds on (most cited first, 6 of 6)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review](https://mlsanity.com/paper/2004.05224/) (2020)
- [On the Evaluation of Conditional GANs](https://mlsanity.com/paper/1907.08175/) (2019)
- [Applying Deep Learning to Specific Learning Disorder Screening](https://mlsanity.com/paper/2008.13525/) (2020)

## Built on by (most cited first, 2 of 2)

- [Depth- and Semantics-aware Multi-modal Domain Translation: Generating 3D Panoramic Color Images from LiDAR Point Clouds](https://mlsanity.com/paper/2302.07661/) (2023)
- [Generating Clear Images From Images With Distortions Caused by Adverse Weather Using Generative Adversarial Networks](https://mlsanity.com/paper/2211.05234/) (2022)

## Abstract (arXiv)

We examined the feasibility of generative adversarial networks (GANs) to generate photo-realistic images from LiDAR point clouds. For this purpose, we created a dataset of point cloud image pairs and trained the GAN to predict photorealistic images from LiDAR point clouds containing reflectance and distance information. Our models learned how to predict realistically looking images from just point cloud data, even images with black cars. Black cars are difficult to detect directly from point clouds because of their low level of reflectivity. This approach might be used in the future to perform visual object recognition on photorealistic images generated from LiDAR point clouds. In addition to the conventional LiDAR system, a second system that generates photorealistic images from LiDAR point clouds would run simultaneously for visual object recognition in real-time. In this way, we might preserve the supremacy of LiDAR and benefit from using photo-realistic images for visual object recognition without the usage of any camera. In addition, this approach could be used to colorize point clouds without the usage of any camera images.