# Minimum Description Length Revisited

arXiv: https://arxiv.org/abs/1908.08484 · HTML: https://mlsanity.com/paper/1908.08484/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2019.

## Built on by (most cited first, 30 of 37)

- [The Local Learning Coefficient: A Singularity-Aware Complexity Measure](https://mlsanity.com/paper/2308.12108/) (2023)
- [Causal Discovery from Temporal Data: An Overview and New Perspectives](https://mlsanity.com/paper/2303.10112/) (2023)
- [Exhaustive Symbolic Regression](https://mlsanity.com/paper/2211.11461/) (2022)
- [The no-free-lunch theorems of supervised learning](https://mlsanity.com/paper/2202.04513/) (2022)
- [First Steps of an Approach to the ARC Challenge based on Descriptive Grid Models and the Minimum Description Length Principle](https://mlsanity.com/paper/2112.00848/) (2021)
- [Causal Discovery in Hawkes Processes by Minimum Description Length](https://mlsanity.com/paper/2206.06124/) (2022)
- [Minimum Description Length and Generalization Guarantees for Representation Learning](https://mlsanity.com/paper/2402.03254/) (2024)
- [Priors for symbolic regression](https://mlsanity.com/paper/2304.06333/) (2023)
- [Tackling the Abstraction and Reasoning Corpus (ARC) with Object-centric Models and the MDL Principle](https://mlsanity.com/paper/2311.00545/) (2023)
- [Truly Unordered Probabilistic Rule Sets for Multi-class Classification](https://mlsanity.com/paper/2206.08804/) (2022)
- [Batch Bayesian Optimization on Permutations using the Acquisition Weighted Kernel](https://mlsanity.com/paper/2102.13382/) (2021)
- [Learning Logic Programs From Noisy Failures](https://mlsanity.com/paper/2201.03702/) (2021)
- [Compressibility Measures Complexity: Minimum Description Length Meets Singular Learning Theory](https://mlsanity.com/paper/2510.12077/) (2025)
- [OneLatent: Single-Token Compression for Visual Latent Reasoning](https://mlsanity.com/paper/2602.13738/) (2026)
- [Robust subgroup discovery](https://mlsanity.com/paper/2103.13686/) (2021)
- [Conditional Density Estimation with Histogram Trees](https://mlsanity.com/paper/2410.11449/) (2024)
- [Granger Causal Inference in Multivariate Hawkes Processes by Minimum Message Length](https://mlsanity.com/paper/2309.02027/) (2023)
- [Generating Interpretable Networks using Hypernetworks](https://mlsanity.com/paper/2312.03051/) (2023)
- [Foundation of Calculating Normalized Maximum Likelihood for Continuous Probability Models](https://mlsanity.com/paper/2409.08387/) (2024)
- [Constraining the outputs of ReLU neural networks](https://mlsanity.com/paper/2508.03867/) (2025)
- [The Calibration Turn in AI-Assisted Research: A Conceptual and Methodological Framework for Evidence-Licensed Claims](https://mlsanity.com/paper/2606.31273/) (2026)
- [Prequential MDL for Causal Structure Learning with Neural Networks](https://mlsanity.com/paper/2107.05481/) (2021)
- [(Exhaustive) Symbolic Regression and model selection by minimum description length](https://mlsanity.com/paper/2507.13033/) (2025)
- [Hallucination is a Consequence of Space-Optimality: A Rate-Distortion Theorem for Membership Testing](https://mlsanity.com/paper/2602.00906/) (2026)
- [A Pluggable Learned Index Method via Sampling and Gap Insertion](https://mlsanity.com/paper/2101.00808/) (2021)
- [Far from Asymptopia](https://mlsanity.com/paper/2205.03343/) (2022)
- [Multiclass classification utilising an estimated algorithmic probability prior](https://mlsanity.com/paper/2212.07426/) (2022)
- [Probabilistic Truly Unordered Rule Sets](https://mlsanity.com/paper/2401.09918/) (2024)
- [From Mechanistic to Compositional Interpretability](https://mlsanity.com/paper/2605.08934/) (2026)
- [The Normalized Maximum Likelihood for Regular Non-Smooth Models: Measure-Theoretic Foundations and Geometric Sampling](https://mlsanity.com/paper/2605.24477/) (2026)

## Abstract (arXiv)

This is an up-to-date introduction to and overview of the Minimum Description Length (MDL) Principle, a theory of inductive inference that can be applied to general problems in statistics, machine learning and pattern recognition. While MDL was originally based on data compression ideas, this introduction can be read without any knowledge thereof. It takes into account all major developments since 2007, the last time an extensive overview was written. These include new methods for model selection and averaging and hypothesis testing, as well as the first completely general definition of em MDL estimators. Incorporating these developments, MDL can be seen as a powerful extension of both penalized likelihood and Bayesian approaches, in which penalization functions and prior distributions are replaced by more general luckiness functions, average-case methodology is replaced by a more robust worst-case approach, and in which methods classically viewed as highly distinct, such as AIC vs BIC and cross-validation vs Bayes can, to a large extent, be viewed from a unified perspective.