# The Implications of the No-Free-Lunch Theorems for Meta-induction

arXiv: https://arxiv.org/abs/2103.11956 · HTML: https://mlsanity.com/paper/2103.11956/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 39 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 3 of 3)

- [The ground truth about metadata and community detection in networks](https://mlsanity.com/paper/1608.05878/) (2016)
- [Using stacking to average Bayesian predictive distributions](https://mlsanity.com/paper/1704.02030/) (2017)
- [Stacking Models for Nearly Optimal Link Prediction in Complex Networks](https://mlsanity.com/paper/1909.07578/) (2019)

## Built on by (most cited first, 6 of 6)

- [Toward Universal Laws of Outlier Propagation](https://mlsanity.com/paper/2502.08593/) (2025)
- [What can we know about that which we cannot even imagine?](https://mlsanity.com/paper/2208.03886/) (2022)
- [Towards the Best Solution for Complex System Reliability: Can Statistics Outperform Machine Learning?](https://mlsanity.com/paper/2410.04238/) (2024)
- [Coordination Requires Simplification: Thermodynamic Bounds on Multi-Objective Compromise in Natural and Artificial Intelligence](https://mlsanity.com/paper/2509.23144/) (2025)
- [Empirical Evaluation of No Free Lunch Violations in Permutation-Based Optimization](https://mlsanity.com/paper/2603.03613/) (2026)
- [Understanding Generalization Requires Universal Induction](https://mlsanity.com/paper/2609.34458/) (2026)

## Abstract (arXiv)

The important recent book by G. Schurz appreciates that the no-free-lunch theorems (NFL) have major implications for the problem of (meta) induction. Here I review the NFL theorems, emphasizing that they do not only concern the case where there is a uniform prior -- they prove that there are "as many priors" (loosely speaking) for which any induction algorithm A out-generalizes some induction algorithm B as vice-versa. Importantly though, in addition to the NFL theorems, there are many free lunch theorems. In particular, the NFL theorems can only be used to compare the marginal expected performance of an induction algorithm A with the marginal expected performance of an induction algorithm B. There is a rich set of free lunches which instead concern the statistical correlations among the generalization errors of induction algorithms. As I describe, the meta-induction algorithms that Schurz advocate as a "solution to Hume's problem" are just an example of such a free lunch based on correlations among the generalization errors of induction algorithms. I end by pointing out that the prior that Schurz advocates, which is uniform over bit frequencies rather than bit patterns, is contradicted by thousands of experiments in statistical physics and by the great success of the maximum entropy procedure in inductive inference.