mlsanityData to 14 Oct 2026

Autoencoding Under Normalization Constraints

arXiv 2105.05735 ↗First posted 12 May 2021cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

Likelihood is a standard estimate for outlier detection. The specific role of the normalization constraint is to ensure that the out-of-distribution (OOD) regime has a small likelihood when samples are learned using maximum likelihood. Because autoencoders do not possess such a process of normalization, they often fail to recognize outliers even when they are obviously OOD. We propose the Normalized Autoencoder (NAE), a normalized probabilistic model constructed from an autoencoder. The probability density of NAE is defined using the reconstruction error of an autoencoder, which is differently defined in the conventional energy-based model. In our model, normalization is enforced by suppressing the reconstruction of negative samples, significantly improving the outlier detection performance. Our experimental results confirm the efficacy of NAE, both in detecting outliers and in generating in-distribution samples.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 36 of its 68 references that are in the map. First posted on arXiv 12 May 2021.

Builds on

36 of 68 references.

References · 36

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Neural Discrete Representation Learningcited by 4,712
  3. 2013Auto-Encoding Variational Bayescited by 4,053
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  5. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  6. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  7. 2018Group Normalizationcited by 1,524
  8. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  9. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  10. 2016Pixel Recurrent Neural Networkscited by 816
  11. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
  12. 2019Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detectioncited by 564
24 more
  1. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 453
  2. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  3. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 417
  4. 2018GANomaly: Semi-Supervised Anomaly Detection via Adversarial Trainingcited by 409
  5. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 405
  6. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 284
  7. 2021How to Train Your Energy-Based Modelscited by 274
  8. 2018Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoderscited by 264
  9. 2018Hyperspherical Variational Auto-Encoderscited by 189
  10. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  11. 2016A Theory of Generative ConvNetcited by 174
  12. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 148
  13. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  14. 2019From Variational to Deterministic Autoencoderscited by 133
  15. 2016Energy-based Generative Adversarial Networkcited by 119
  16. 2020Flows for simultaneous manifold learning and density estimationcited by 118
  17. 2020VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Modelscited by 104
  18. 2018Generative Probabilistic Novelty Detection with Adversarial Autoencoderscited by 99
  19. 2018Spherical Latent Spaces for Stable Variational Autoencoderscited by 67
  20. 2020Perfect density models cannot guarantee anomaly detectioncited by 39
  21. 2019Autoregressive Energy Machinescited by 23
  22. 2018Anomaly Detection for Skin Disease Images Using Variational Autoencodercited by 20
  23. 2020Autoregressive Score Matchingcited by 15
  24. 2019Latent Variables on Spheres for Autoencoders in High Dimensionscited by 7

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 4; 2023: 14; 2024: 5; 2025: 1.

26 papers.

Sort:

By area:

  1. 2023Mask, Stitch, and Re-Sample: Enhancing Robustness and Generalizability in Anomaly Detection through Automatic Diffusion Modelscited by 50
  2. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  3. 2023Machine Learning for Anomaly Detection in Particle Physicscited by 30
  4. 2024A Geometric Explanation of the Likelihood OOD Detection Paradoxcited by 22
  5. 2023Energy-Based Models for Anomaly Detection: A Manifold Diffusion Recovery Approachcited by 18
  6. 2023A Review of Change of Variable Formulas for Generative Modelingcited by 13
  7. 2025Autoencoders for Anomaly Detection are Unreliablecited by 9
  8. 2024"Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihoodcited by 8
  9. 2021Normality-Calibrated Autoencoder for Unsupervised Anomaly Detection on Data Contaminationcited by 5
  10. 2021Data Invariants to Understand Unsupervised Out-of-Distribution Detectioncited by 5
  11. 2022Neural Implicit Manifold Learning for Topology-Aware Density Estimationcited by 5
  12. 2023MMP++: Motion Manifold Primitives with Parametric Curve Modelscited by 5
14 more
  1. 2023An Iterative Method for Unsupervised Robust Anomaly Detection Under Data Contaminationcited by 4
  2. 2024Maximum Entropy Inverse Reinforcement Learning of Diffusion Models with Energy-Based Modelscited by 4
  3. 2022Positive Difference Distribution for Image Outlier Detection using Normalizing Flows and Contrastive Datacited by 3
  4. 2023Versatile Energy-Based Probabilistic Models for High Energy Physicscited by 2
  5. 2023TR0N: Translator Networks for 0-Shot Plug-and-Play Conditional Generationcited by 2
  6. 2023Restricted Generative Projection for One-Class Classification and Anomaly Detectioncited by 2
  7. 2024Deep Positive-Unlabeled Anomaly Detection for Contaminated Unlabeled Datacited by 2
  8. 2022Geometric instability of out of distribution data across autoencoder architecturecited by 1
  9. 2022Evaluating Out-of-Distribution Detectors Through Adversarial Generation of Outlierscited by 1
  10. 2023How to Construct Energy for Images? Denoising Autoencoder Can Be Energy Based Modelcited by 1
  11. 2023Triggering Dark Showers with Conditional Dual Auto-Encoderscited by 1
  12. 2023Learning Nonparametric High-Dimensional Generative Models: The Empirical-Beta-Copula Autoencodercited by 0
  13. 2023Adversarial Anomaly Detection using Gaussian Priors and Nonlinear Anomaly Scorescited by 0
  14. 2023Generalized Contrastive Divergence: Joint Training of Energy-Based Model and Diffusion Model through Inverse Reinforcement Learningcited by 0