# Out-of-Distribution Detection using Maximum Entropy Coding

arXiv: https://arxiv.org/abs/2404.17023 · HTML: https://mlsanity.com/paper/2404.17023/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 39 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Builds on (most cited first, 8 of 8)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [One-Class Support Measure Machines for Group Anomaly Detection](https://mlsanity.com/paper/1303.0309/) (2013)
- [Group Anomaly Detection using Deep Generative Models](https://mlsanity.com/paper/1804.04876/) (2018)
- [Accelerated Computation of a High Dimensional Kolmogorov-Smirnov Distance](https://mlsanity.com/paper/2106.13706/) (2021)
- [Detecting Change Signs with Differential MDL Change Statistics for COVID-19 Pandemic Analysis](https://mlsanity.com/paper/2007.15179/) (2020)
- [Graph Coding for Model Selection and Anomaly Detection in Gaussian Graphical Models](https://mlsanity.com/paper/2102.02431/) (2021)

## Abstract (arXiv)

Given a default distribution P and a set of test data xM=x1,x2,ldots,xM this paper seeks to answer the question if it was likely that xM was generated by P. For discrete distributions, the definitive answer is in principle given by Kolmogorov-Martin-L"of randomness. In this paper we seek to generalize this to continuous distributions. We consider a set of statistics T1(xM),T2(xM),ldots. To each statistic we associate its maximum entropy distribution and with this a universal source coder. The maximum entropy distributions are subsequently combined to give a total codelength, which is compared with -log P(xM). We show that this approach satisfied a number of theoretical properties. For real world data P usually is unknown. We transform data into a standard distribution in the latent space using a bidirectional generate network and use maximum entropy coding there. We compare the resulting method to other methods that also used generative neural networks to detect anomalies. In most cases, our results show better performance.