mlsanityData to 7 Oct 2026

Improved Autoregressive Modeling with Distribution Smoothing

arXiv 2103.15089 ↗First posted 28 Mar 2021cs.LG cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

While autoregressive models excel at image compression, their sample quality is often lacking. Although not realistic, generated images often have high likelihood according to the model, resembling the case of adversarial examples. Inspired by a successful adversarial defense method, we incorporate randomized smoothing into autoregressive generative modeling. We first model a smoothed version of the data distribution, and then reverse the smoothing process to recover the original data distribution. This procedure drastically improves the sample quality of existing autoregressive models on several synthetic and real-world image datasets while obtaining competitive likelihoods on synthetic datasets.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 27 of its 48 references that are in the map. First posted on arXiv 28 Mar 2021.

Builds on

27 of 48 references.

References · 27

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2015Variational Inference with Normalizing Flowscited by 2,161
  8. 2016Density estimation using Real NVPcited by 2,029
  9. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  10. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  11. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  12. 2018Demystifying MMD GANscited by 1,268
15 more
  1. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  2. 2016Pixel Recurrent Neural Networkscited by 814
  3. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  4. 2018Joint Autoregressive and Hierarchical Priors for Learned Image Compressioncited by 642
  5. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  6. 2018Invertible Residual Networkscited by 364
  7. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
  8. 2015A note on the evaluation of generative modelscited by 291
  9. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  10. 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
  11. 2018Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscalingcited by 74
  12. 2013RNADE: The real-valued neural autoregressive density-estimatorcited by 65
  13. 2019Neural Empirical Bayescited by 46
  14. 2013Deep Generative Stochastic Networks Trainable by Backpropcited by 36
  15. 2018Autoregressive Quantile Networks for Generative Modelingcited by 31

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 8; 2023: 2; 2024: 0; 2025: 2.

18 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  2. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  3. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  4. 2021On Fast Sampling of Diffusion Probabilistic Modelscited by 170
  5. 2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
  6. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 113
  7. 2021Diffusion Normalizing Flowcited by 77
  8. 2022Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoderscited by 42
  9. 2022Diagnosing and Fixing Manifold Overfitting in Deep Generative Modelscited by 28
  10. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  11. 2021A likelihood approach to nonparametric estimation of a singular distribution using deep generative modelscited by 12
  12. 2025Ideas in Inference-time Scaling can Benefit Generative Pre-training Algorithmscited by 12
6 more
  1. 2021Style Equalization: Unsupervised Learning of Controllable Generative Sequence Modelscited by 11
  2. 2022Out-of-Distribution Detection with Reconstruction Error and Typicality-based Penaltycited by 11
  3. 2022Denoising Deep Generative Modelscited by 6
  4. 2022Optimizing Hierarchical Image VAEs for Sample Qualitycited by 5
  5. 2023One-Line-of-Code Data Mollification Improves Optimization of Likelihood-based Generative Modelscited by 4
  6. 2021A Generic Approach for Enhancing GANs by Regularized Latent Optimizationcited by 0