# Quantifying point cloud realism through adversarially learned latent representations

arXiv: https://arxiv.org/abs/2109.11775 · HTML: https://mlsanity.com/paper/2109.11775/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 28 of 28)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [Pros and Cons of GAN Evaluation Measures](https://mlsanity.com/paper/1802.03446/) (2018)
- [VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning](https://mlsanity.com/paper/1705.07761/) (2017)
- [3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions](https://mlsanity.com/paper/1905.06292/) (2019)
- [Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations](https://mlsanity.com/paper/1707.00075/) (2017)
- [Mode Regularized Generative Adversarial Networks](https://mlsanity.com/paper/1612.02136/) (2016)
- [PacGAN: The power of two samples in generative adversarial networks](https://mlsanity.com/paper/1712.04086/) (2017)
- [Disentangling factors of variation in deep representations using adversarial training](https://mlsanity.com/paper/1611.03383/) (2016)
- [An empirical study on evaluation metrics of generative adversarial networks](https://mlsanity.com/paper/1806.07755/) (2018)
- [DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data](https://mlsanity.com/paper/1706.02071/) (2017)
- [Deep Generative Modeling of LiDAR Data](https://mlsanity.com/paper/1812.01180/) (2018)
- [On GANs and GMMs](https://mlsanity.com/paper/1805.12462/) (2018)
- [Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.04357/) (2016)
- [LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation](https://mlsanity.com/paper/1703.01560/) (2017)
- [AdaGAN: Boosting Generative Models](https://mlsanity.com/paper/1701.02386/) (2017)
- [LiDAR Sensor modeling and Data augmentation with GANs for Autonomous driving](https://mlsanity.com/paper/1905.07290/) (2019)
- [RenderGAN: Generating Realistic Labeled Data](https://mlsanity.com/paper/1611.01331/) (2016)
- [Gradient Reversal Against Discrimination](https://mlsanity.com/paper/1807.00392/) (2018)
- [Generating images with recurrent adversarial networks](https://mlsanity.com/paper/1602.05110/) (2016)
- [CNN-based synthesis of realistic high-resolution LiDAR data](https://mlsanity.com/paper/1907.00787/) (2019)
- [Decoupled Learning for Conditional Adversarial Networks](https://mlsanity.com/paper/1801.06790/) (2018)

## Built on by (most cited first, 2 of 2)

- [Survey on LiDAR Perception in Adverse Weather Conditions](https://mlsanity.com/paper/2304.06312/) (2023)
- [A Realism Metric for Generated LiDAR Point Clouds](https://mlsanity.com/paper/2208.14958/) (2022)

## Abstract (arXiv)

Judging the quality of samples synthesized by generative models can be tedious and time consuming, especially for complex data structures, such as point clouds. This paper presents a novel approach to quantify the realism of local regions in LiDAR point clouds. Relevant features are learned from real-world and synthetic point clouds by training on a proxy classification task. Inspired by fair networks, we use an adversarial technique to discourage the encoding of dataset-specific information. The resulting metric can assign a quality score to samples without requiring any task specific annotations. In a series of experiments, we confirm the soundness of our metric by applying it in controllable task setups and on unseen data. Additional experiments show reliable interpolation capabilities of the metric between data with varying degree of realism. As one important application, we demonstrate how the local realism score can be used for anomaly detection in point clouds.