# Far from Asymptopia

arXiv: https://arxiv.org/abs/2205.03343 · HTML: https://mlsanity.com/paper/2205.03343/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 57 references that are in the map.
- First posted on arXiv 6 May 2022.

## Builds on (most cited first, 5 of 5)

- [Minimum Description Length Revisited](https://mlsanity.com/paper/1908.08484/) (2019)
- [Pathological spectra of the Fisher information metric and its variants in deep neural networks](https://mlsanity.com/paper/1910.05992/) (2019)
- [Maximizing the information learned from finite data selects a simple model](https://mlsanity.com/paper/1705.01166/) (2017)
- [Learning Approximately Objective Priors](https://mlsanity.com/paper/1704.01168/) (2017)
- [Iterative Markov Chain Monte Carlo Computation of Reference Priors and Minimax Risk](https://mlsanity.com/paper/1301.2286/) (2013)

## Built on by (most cited first, 1 of 1)

- [NCoder -- A Quantum Field Theory approach to encoding data](https://mlsanity.com/paper/2402.00944/) (2024)

## Abstract (arXiv)

Inference from limited data requires a notion of measure on parameter space, most explicit in the Bayesian framework as a prior. Here we demonstrate that Jeffreys prior, the best-known uninformative choice, introduces enormous bias when applied to typical scientific models. Such models have a relevant effective dimensionality much smaller than the number of microscopic parameters. Because Jeffreys prior treats all microscopic parameters equally, it is from uniform when projected onto the sub-space of relevant parameters, due to variations in the local co-volume of irrelevant directions. We present results on a principled choice of measure which avoids this issue, leading to unbiased inference in complex models. This optimal prior depends on the quantity of data to be gathered, and approaches Jeffreys prior in the asymptotic limit. However, this limit cannot be justified without an impossibly large amount of data, exponential in the number of microscopic parameters.