# Latent Space Refinement for Deep Generative Models

arXiv: https://arxiv.org/abs/2106.00792 · HTML: https://mlsanity.com/paper/2106.00792/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 97 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 30 of 42)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [Diagnosing and Enhancing VAE Models](https://mlsanity.com/paper/1903.05789/) (2019)
- [Flow-based generative models for Markov chain Monte Carlo in lattice field theory](https://mlsanity.com/paper/1904.12072/) (2019)
- [Flows for simultaneous manifold learning and density estimation](https://mlsanity.com/paper/2003.13913/) (2020)
- [Equivariant flow-based sampling for lattice gauge theory](https://mlsanity.com/paper/2003.06413/) (2020)
- [Iterative Amortized Inference](https://mlsanity.com/paper/1807.09356/) (2018)
- [Semi-Amortized Variational Autoencoders](https://mlsanity.com/paper/1802.02550/) (2018)
- [CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks](https://mlsanity.com/paper/1712.10321/) (2017)
- [Anomaly Detection with Density Estimation](https://mlsanity.com/paper/2001.04990/) (2020)
- [Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling](https://mlsanity.com/paper/2003.06060/) (2020)
- [Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models](https://mlsanity.com/paper/2002.07101/) (2020)
- [Accelerating Science with Generative Adversarial Networks: An Application to 3D Particle Showers in Multi-Layer Calorimeters](https://mlsanity.com/paper/1705.02355/) (2017)
- [Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis](https://mlsanity.com/paper/1701.05927/) (2017)
- [Calorimetry with Deep Learning: Particle Simulation and Reconstruction for Collider Physics](https://mlsanity.com/paper/1912.06794/) (2019)
- [Variational Autoencoders for Anomalous Jet Tagging](https://mlsanity.com/paper/2007.01850/) (2020)
- [OmniFold: A Method to Simultaneously Unfold All Observables](https://mlsanity.com/paper/1911.09107/) (2019)
- [How to GAN away Detector Effects](https://mlsanity.com/paper/1912.00477/) (2019)
- [LHC analysis-specific datasets with Generative Adversarial Networks](https://mlsanity.com/paper/1901.05282/) (2019)
- [i-flow: High-dimensional Integration and Sampling with Normalizing Flows](https://mlsanity.com/paper/2001.05486/) (2020)
- [Generative Models for Fast Calorimeter Simulation.LHCb case](https://mlsanity.com/paper/1812.01319/) (2018)
- [Controlling Physical Attributes in GAN-Accelerated Simulation of Electromagnetic Calorimeters](https://mlsanity.com/paper/1711.08813/) (2017)
- [DCTRGAN: Improving the Precision of Generative Models with Reweighting](https://mlsanity.com/paper/2009.03796/) (2020)
- [Neural Networks for Full Phase-space Reweighting and Parameter Tuning](https://mlsanity.com/paper/1907.08209/) (2019)
- [Bump Hunting in Latent Space](https://mlsanity.com/paper/2103.06595/) (2021)

## Built on by (most cited first, 10 of 10)

- [CaloFlow: Fast and Accurate Generation of Calorimeter Showers with Normalizing Flows](https://mlsanity.com/paper/2106.05285/) (2021)
- [Generative Networks for Precision Enthusiasts](https://mlsanity.com/paper/2110.13632/) (2021)
- [Deep Generative Models for Detector Signature Simulation: A Taxonomic Review](https://mlsanity.com/paper/2312.09597/) (2023)
- [New directions for surrogate models and differentiable programming for High Energy Physics detector simulation](https://mlsanity.com/paper/2203.08806/) (2022)
- [High-dimensional and Permutation Invariant Anomaly Detection](https://mlsanity.com/paper/2306.03933/) (2023)
- [Calorimeter shower superresolution](https://mlsanity.com/paper/2308.11700/) (2023)
- [ELSA -- Enhanced latent spaces for improved collider simulations](https://mlsanity.com/paper/2305.07696/) (2023)
- [Calibrating Bayesian Generative Machine Learning for Bayesiamplification](https://mlsanity.com/paper/2408.00838/) (2024)
- [Steering Language Generation: Harnessing Contrastive Expert Guidance and Negative Prompting for Coherent and Diverse Synthetic Data Generation](https://mlsanity.com/paper/2308.07645/) (2023)
- [Extrapolating Jet Radiation with Autoregressive Transformers](https://mlsanity.com/paper/2412.12074/) (2024)

## Abstract (arXiv)

Deep generative models are becoming widely used across science and industry for a variety of purposes. A common challenge is achieving a precise implicit or explicit representation of the data probability density. Recent proposals have suggested using classifier weights to refine the learned density of deep generative models. We extend this idea to all types of generative models and show how latent space refinement via iterated generative modeling can circumvent topological obstructions and improve precision. This methodology also applies to cases were the target model is non-differentiable and has many internal latent dimensions which must be marginalized over before refinement. We demonstrate our Latent Space Refinement (LaSeR) protocol on a variety of examples, focusing on the combinations of Normalizing Flows and Generative Adversarial Networks.