# Compression, Regularity, Randomness and Emergent Structure: Rethinking Physical Complexity in the Data-Driven Era

arXiv: https://arxiv.org/abs/2505.07222 · HTML: https://mlsanity.com/paper/2505.07222/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 127 references that are in the map.
- First posted on arXiv 12 May 2025.

## Builds on (most cited first, 20 of 20)

- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Modern Koopman Theory for Dynamical Systems](https://mlsanity.com/paper/2102.12086/) (2021)
- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [Linearly-Recurrent Autoencoder Networks for Learning Dynamics](https://mlsanity.com/paper/1712.01378/) (2017)
- [Gaussian Process Prior Variational Autoencoders](https://mlsanity.com/paper/1810.11738/) (2018)
- [Large-scale directed network inference with multivariate transfer entropy and hierarchical statistical testing](https://mlsanity.com/paper/1902.06828/) (2019)
- [Dynamic Balance of Excitation and Inhibition in Human and Monkey Neocortex](https://mlsanity.com/paper/1410.2610/) (2014)
- [Transfer Entropy reconstruction and labeling of neuronal connections from simulated calcium imaging](https://mlsanity.com/paper/1309.4287/) (2013)
- [Blind Construction of Optimal Nonlinear Recursive Predictors for Discrete Sequences](https://mlsanity.com/paper/1408.2025/) (2014)
- [The Origins of Computational Mechanics: A Brief Intellectual History and Several Clarifications](https://mlsanity.com/paper/1710.06832/) (2017)
- [Symmetry's Edge in Cortical Dynamics: Multiscale Dynamics of Ensemble Excitation and Inhibition](https://mlsanity.com/paper/2306.11965/) (2023)

## Abstract (arXiv)

Complexity science offers a wide range of measures for quantifying unpredictability, structure, and information. Yet, a systematic conceptual organization of these measures is still missing. We present a unified framework that locates statistical, algorithmic, and dynamical measures along three axes (regularity, randomness, and complexity) and situates them in a common conceptual space. We map statistical, algorithmic, and dynamical measures into this conceptual space, discussing their computational accessibility and approximability. This taxonomy reveals the deep challenges posed by uncomputability and highlights the emergence of modern data-driven methods (including autoencoders, latent dynamical models, symbolic regression, and physics-informed neural networks) as pragmatic approximations to classical complexity ideals. Latent spaces emerge as operational arenas where regularity extraction, noise management, and structured compression converge, bridging theoretical foundations with practical modeling in high-dimensional systems. We close by outlining implications for physics-informed AI and AI-guided discovery in complex physical systems, arguing that classical questions of complexity remain central to next-generation scientific modeling.