mlsanityData to 7 Oct 2026

On the Out-of-distribution Generalization of Probabilistic Image Modelling

arXiv 2109.02639 ↗First posted 4 Sep 2021cs.CV cs.LG stat.APcited by 32

Not on the map. Why?

Abstract (arXiv)

Out-of-distribution (OOD) detection and lossless compression constitute two problems that can be solved by the training of probabilistic models on a first dataset with subsequent likelihood evaluation on a second dataset, where data distributions differ. By defining the generalization of probabilistic models in terms of likelihood we show that, in the case of image models, the OOD generalization ability is dominated by local features. This motivates our proposal of a Local Autoregressive model that exclusively models local image features towards improving OOD performance. We apply the proposed model to OOD detection tasks and achieve state-of-the-art unsupervised OOD detection performance without the introduction of additional data. Additionally, we employ our model to build a new lossless image compressor: NeLLoC (Neural Local Lossless Compressor) and report state-of-the-art compression rates and model size.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 33 of its 52 references that are in the map. First posted on arXiv 4 Sep 2021.

Builds on

33 of 52 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  7. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  8. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  9. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  10. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  11. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  12. 2016Pixel Recurrent Neural Networkscited by 814
21 more
  1. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  2. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  3. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  4. 2018Deep Learning for Classical Japanese Literaturecited by 415
  5. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 220
  6. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  7. 2018Practical Full Resolution Learned Lossless Image Compressioncited by 93
  8. 2019Practical Lossless Compression with Latent Variables using Bits Back Codingcited by 85
  9. 2019Integer Discrete Flows and Lossless Compressioncited by 82
  10. 2020Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Featurescited by 64
  11. 2019Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variablescited by 51
  12. 2019Detecting semantic anomaliescited by 50
  13. 2018Uncertainty in the Variational Information Bottleneckcited by 39
  14. 2019HiLLoC: Lossless Image Compression with Hierarchical Latent Variable Modelscited by 36
  15. 2020Multiscale Score Matching for Out-of-Distribution Detectioncited by 36
  16. 2020IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compressioncited by 35
  17. 2019Compression with Flows via Local Bits-Back Codingcited by 34
  18. 2019Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matricescited by 32
  19. 2019Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicalitycited by 30
  20. 2020Reducing the Computational Cost of Deep Generative Models with Binary Neural Networkscited by 6
  21. 2019Understanding the (un)interpretability of natural image distributions using generative modelscited by 4

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 15; 2023: 2; 2024: 11; 2025: 2.

32 papers.

Sort:

By area:

  1. 2022NIRVANA: Neural Implicit Representations of Videos with Adaptive Networks and Autoregressive Patch-wise Modelingcited by 50
  2. 2022Lossy Image Compression with Quantized Hierarchical VAEscited by 32
  3. 2024Domain Generalization through Meta-Learning: A Surveycited by 28
  4. 2022Augmenting Softmax Information for Selective Classification with Out-of-Distribution Datacited by 27
  5. 2022Deep Lossy Plus Residual Coding for Lossless and Near-lossless Image Compressioncited by 21
  6. 2024Tackling Structural Hallucination in Image Translation with Local Diffusioncited by 21
  7. 2021Lossless Compression with Probabilistic Circuitscited by 20
  8. 2022CLUTR: Curriculum Learning via Unsupervised Task Representation Learningcited by 19
  9. 2022On the Usefulness of Deep Ensemble Diversity for Out-of-Distribution Detectioncited by 14
  10. 2024Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matchingcited by 13
  11. 2024A Likelihood Ratio-Based Approach to Segmenting Unknown Objectscited by 12
  12. 2024Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detectorcited by 11
20 more
  1. 2022Generalization Gap in Amortized Inferencecited by 10
  2. 2023Likelihood-based Out-of-Distribution Detection with Denoising Diffusion Probabilistic Modelscited by 10
  3. 2025Tackling Hallucination from Conditional Models for Medical Image Reconstruction with DynamicDPScited by 10
  4. 2024Latent-INR: A Flexible Framework for Implicit Representations of Videos with Discriminative Semanticscited by 9
  5. 2024Forte : Finding Outliers with Representation Typicality Estimationcited by 9
  6. 2022Falsehoods that ML researchers believe about OOD detectioncited by 8
  7. 2022Improving VAE-based Representation Learningcited by 6
  8. 2024CALLIC: Content Adaptive Learning for Lossless Image Compressioncited by 6
  9. 2022PILC: Practical Image Lossless Compression with an End-to-end GPU Oriented Neural Frameworkcited by 5
  10. 2024Learning Lossless Compression for High Bit-Depth Volumetric Medical Imagecited by 5
  11. 2022Parallel Neural Local Lossless Compressioncited by 4
  12. 2022Vision Checklist: Towards Testable Error Analysis of Image Models to Help System Designers Interrogate Model Capabilitiescited by 4
  13. 2022Out-of-Distribution Detection with Class Ratio Estimationcited by 4
  14. 2024Approximations to the Fisher Information Metric of Deep Generative Models for Out-Of-Distribution Detectioncited by 2
  15. 2024High-Frequency Enhanced Hybrid Neural Representation for Video Compressioncited by 2
  16. 2023SR-OOD: Out-of-Distribution Detection via Sample Repairingcited by 1
  17. 2025SIEDD: Shared-Implicit Encoder with Discrete Decoderscited by 1
  18. 2021A benchmark with decomposed distribution shifts for 360 monocular depth estimationcited by 0
  19. 2022Multi-Scale Architectures Matter: On the Adversarial Robustness of Flow-based Lossless Compressioncited by 0
  20. 2022Verified Reversible Programming for Verified Lossless Compressioncited by 0