# Bridging Algorithmic Information Theory and Machine Learning: A New Approach to Kernel Learning

arXiv: https://arxiv.org/abs/2311.12624 · HTML: https://mlsanity.com/paper/2311.12624/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 44 references that are in the map.
- First posted on arXiv 21 Nov 2023.

## Builds on (most cited first, 12 of 12)

- [Solving and Learning Nonlinear PDEs with Gaussian Processes](https://mlsanity.com/paper/2103.12959/) (2021)
- [Data-driven approximation of the Koopman generator: Model reduction, system identification, and control](https://mlsanity.com/paper/1909.10638/) (2019)
- [Kernel Flows: from learning kernels from data into the abyss](https://mlsanity.com/paper/1808.04475/) (2018)
- [Kernel-based approximation of the Koopman generator and Schr"odinger operator](https://mlsanity.com/paper/2005.13231/) (2020)
- [Computational Graph Completion](https://mlsanity.com/paper/2110.10323/) (2021)
- [Data-driven geophysical forecasting: Simple, low-cost, and accurate baselines with kernel methods](https://mlsanity.com/paper/2103.10935/) (2021)
- [Learning Dynamical Systems from Data: A Simple Cross-Validation Perspective, Part V: Sparse Kernel Flows for 132 Chaotic Dynamical Systems](https://mlsanity.com/paper/2301.10321/) (2023)
- [Learning dynamical systems from data: A simple cross-validation perspective, part III: Irregularly-Sampled Time Series](https://mlsanity.com/paper/2111.13037/) (2021)
- [Kernel methods for center manifold approximation and a data-based version of the Center Manifold Theorem](https://mlsanity.com/paper/2012.00338/) (2020)
- [Dimensionality Reduction of Complex Metastable Systems via Kernel Embeddings of Transition Manifolds](https://mlsanity.com/paper/1904.08622/) (2019)
- [A Note on Kernel Methods for Multiscale Systems with Critical Transitions](https://mlsanity.com/paper/1804.09415/) (2018)
- [Learning "best" kernels from data in Gaussian process regression. With application to aerodynamics](https://mlsanity.com/paper/2206.02563/) (2022)

## Built on by (most cited first, 3 of 3)

- [Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformers](https://mlsanity.com/paper/2509.22445/) (2025)
- [Adaptive Kernel Selection for Kernelized Diffusion Maps](https://mlsanity.com/paper/2604.18402/) (2026)
- [Dictionary learning for Kernel EDMD](https://mlsanity.com/paper/2604.25572/) (2026)

## Abstract (arXiv)

Machine Learning (ML) and Algorithmic Information Theory (AIT) look at Complexity from different points of view. We explore the interface between AIT and Kernel Methods (that are prevalent in ML) by adopting an AIT perspective on the problem of learning kernels from data, in kernel ridge regression, through the method of Sparse Kernel Flows. In particular, by looking at the differences and commonalities between Minimal Description Length (MDL) and Regularization in Machine Learning (RML), we prove that the method of Sparse Kernel Flows is the natural approach to adopt to learn kernels from data. This approach aligns naturally with the MDL principle, offering a more robust theoretical basis than the existing reliance on cross-validation. The study reveals that deriving Sparse Kernel Flows does not require a statistical approach; instead, one can directly engage with code-lengths and complexities, concepts central to AIT. Thereby, this approach opens the door to reformulating algorithms in machine learning using tools from AIT, with the aim of providing them a more solid theoretical foundation.