mlsanityData to 14 Oct 2026

Why the Third Axis Is Freedom

arXiv 2608.05423 ↗First posted 5 Aug 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In generative training, a model produces an output and is penalised for its difference from an example. With one output per comparison, a model that produces one common answer can outperform a model retaining a broader repertoire. Explorative Modeling (XM) produces K outputs per comparison and updates on the closest, claiming exploration as a "third pretraining axis" associated with generative expressivity. Here I show the third axis is actually freedom, meaning the weakness of the constraint implied by a model's behaviour. Previous work showed freedom is a property of function rather than form. Parameters, architecture, minimum-description-length (MDL), and data can vary while the behavioural constraint remains unchanged. It was formally proved that weakest models are likeliest to generalise, and freedom selection beat MDL by 110-500% in induction experiments. I prove average XM loss depends on the chance a candidate misses an acceptable region, with exploration raising miss probability to power K. For K>1, match probability rises with freedom. I then demonstrate empirically that XM optimises for freedom. In a Forward XM experiment, larger K increased or saturated measured freedom, and increased freedom at every tested value under context-dependent targets. I trained XM candidate pools and compared validation selection with a freedom selector that read unlabelled parent contexts. Freedom won in 29 of 30 cases. Generative expressivity is a mode-count proxy for freedom, that discards the extension structure that gives freedom its generalisation significance. XM is a means, freedom an end, and selecting for freedom improved XM under distribution shift.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 31 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

8 of 31 references.

References · 8

  1. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  2. 2016Stochastic Multiple Choice Learning for Training Diverse Deep Ensemblescited by 77
  3. 2018Implicit Maximum Likelihood Estimationcited by 52
  4. 2023The Optimal Choice of Hypothesis Is the Weakest, Not the Shortestcited by 12
  5. 2024Is Complexity an Illusion?cited by 5
  6. 2024Are Biological Systems More Intelligent Than Artificial Intelligence?cited by 5
  7. 2026Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generationcited by 4
  8. 2026Are Flat Minima an Illusion?cited by 1

Built on by

Not cited yet.