Why the Third Axis Is Freedom
Not on the map. Why?
Abstract (arXiv)
In generative training, a model produces an output and is penalised for its difference from an example. With one output per comparison, a model that produces one common answer can outperform a model retaining a broader repertoire. Explorative Modeling (XM) produces K outputs per comparison and updates on the closest, claiming exploration as a "third pretraining axis" associated with generative expressivity. Here I show the third axis is actually freedom, meaning the weakness of the constraint implied by a model's behaviour. Previous work showed freedom is a property of function rather than form. Parameters, architecture, minimum-description-length (MDL), and data can vary while the behavioural constraint remains unchanged. It was formally proved that weakest models are likeliest to generalise, and freedom selection beat MDL by 110-500% in induction experiments. I prove average XM loss depends on the chance a candidate misses an acceptable region, with exploration raising miss probability to power K. For K>1, match probability rises with freedom. I then demonstrate empirically that XM optimises for freedom. In a Forward XM experiment, larger K increased or saturated measured freedom, and increased freedom at every tested value under context-dependent targets. I trained XM candidate pools and compared validation selection with a freedom selector that read unlabelled parent contexts. Freedom won in 29 of 30 cases. Generative expressivity is a mode-count proxy for freedom, that discards the extension structure that gives freedom its generalisation significance. XM is a means, freedom an end, and selecting for freedom improved XM under distribution shift.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 31 references that are in the map. First posted on arXiv 5 Aug 2026.
Builds on
8 of 31 references.
References · 8
- 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
- 2016Stochastic Multiple Choice Learning for Training Diverse Deep Ensemblescited by 77
- 2018Implicit Maximum Likelihood Estimationcited by 52
- 2023The Optimal Choice of Hypothesis Is the Weakest, Not the Shortestcited by 12
- 2024Is Complexity an Illusion?cited by 5
- 2024Are Biological Systems More Intelligent Than Artificial Intelligence?cited by 5
- 2026Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generationcited by 4
- 2026Are Flat Minima an Illusion?cited by 1
Built on by
Not cited yet.