# Divergence Frontiers for Generative Models: Sample Complexity, Quantization Effects, and Frontier Integrals

arXiv: https://arxiv.org/abs/2106.07898 · HTML: https://mlsanity.com/paper/2106.07898/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 68 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [Reliable Fidelity and Diversity Metrics for Generative Models](https://mlsanity.com/paper/2002.09797/) (2020)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [Spreading vectors for similarity search](https://mlsanity.com/paper/1806.03198/) (2018)
- [Practical and Consistent Estimation of f-Divergences](https://mlsanity.com/paper/1905.11112/) (2019)
- [Evaluation Metrics for Conditional Image Generation](https://mlsanity.com/paper/2004.12361/) (2020)
- [Non-Asymptotic Performance Guarantees for Neural Estimation of mathsff-Divergences](https://mlsanity.com/paper/2103.06923/) (2021)
- [Quantitatively Evaluating GANs With Divergences Proposed for Training](https://mlsanity.com/paper/1803.01045/) (2018)
- [Deep Residual Mixture Models](https://mlsanity.com/paper/2006.12063/) (2020)

## Built on by (most cited first, 18 of 18)

- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [Remasking Discrete Diffusion Models with Inference-Time Scaling](https://mlsanity.com/paper/2503.00307/) (2025)
- [DistiLLM: Towards Streamlined Distillation for Large Language Models](https://mlsanity.com/paper/2402.03898/) (2024)
- [MAUVE Scores for Generative Models: Theory and Practice](https://mlsanity.com/paper/2212.14578/) (2022)
- [Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling](https://mlsanity.com/paper/2510.01329/) (2025)
- [A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Science](https://mlsanity.com/paper/2403.12636/) (2024)
- [SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation](https://mlsanity.com/paper/2405.10040/) (2024)
- [KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server](https://mlsanity.com/paper/2410.05725/) (2024)
- [InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacy](https://mlsanity.com/paper/2507.02974/) (2025)
- [Enabling Approximate Joint Sampling in Diffusion LMs](https://mlsanity.com/paper/2509.22738/) (2025)
- [RenyiCL: Contrastive Representation Learning with Skew Renyi Divergence](https://mlsanity.com/paper/2208.06270/) (2022)
- [On the Usefulness of Embeddings, Clusters and Strings for Text Generator Evaluation](https://mlsanity.com/paper/2205.16001/) (2022)
- [Fixed-Point Masked Generative Modeling](https://mlsanity.com/paper/2605.31215/) (2026)
- [CorrSynth -- A Correlated Sampling Method for Diverse Dataset Generation from LLMs](https://mlsanity.com/paper/2411.08553/) (2024)
- [Style-News: Incorporating Stylized News Generation and Adversarial Verification for Neural Fake News Detection](https://mlsanity.com/paper/2401.15509/) (2024)
- [Privately Fine-Tuned LLMs Preserve Temporal Dynamics in Tabular Data](https://mlsanity.com/paper/2602.02766/) (2026)
- [Gumbel Distillation for Parallel Text Generation](https://mlsanity.com/paper/2603.22216/) (2026)
- [Token Homogenization under Positional Bias](https://mlsanity.com/paper/2508.17126/) (2025)

## Abstract (arXiv)

The spectacular success of deep generative models calls for quantitative tools to measure their statistical performance. Divergence frontiers have recently been proposed as an evaluation framework for generative models, due to their ability to measure the quality-diversity trade-off inherent to deep generative modeling. We establish non-asymptotic bounds on the sample complexity of divergence frontiers. We also introduce frontier integrals which provide summary statistics of divergence frontiers. We show how smoothed estimators such as Good-Turing or Krichevsky-Trofimov can overcome the missing mass problem and lead to faster rates of convergence. We illustrate the theoretical results with numerical examples from natural language processing and computer vision.