mlsanityData to 14 Oct 2026

auto-fpt: Automating Free Probability Theory Calculations for Machine Learning Theory

arXiv 2504.10754 ↗First posted 14 Apr 2025cs.LG cs.MScited by 0

Not on the map. Why?

Abstract (arXiv)

A large part of modern machine learning theory often involves computing the high-dimensional expected trace of a rational expression of large rectangular random matrices. To symbolically compute such quantities using free probability theory, we introduce auto-fpt, a lightweight Python and SymPy-based tool that can automatically produce a reduced system of fixed-point equations which can be solved for the quantities of interest, and effectively constitutes a theory. We overview the algorithmic ideas underlying auto-fpt and its applications to various interesting problems, such as the high-dimensional error of linearized feed-forward neural networks, recovering well-known results. We hope that auto-fpt streamlines the majority of calculations involved in high-dimensional analysis, while helping the machine learning community reproduce known and uncover new phenomena.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 27 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

15 of 27 references.

References · 15

  1. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  2. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 513
  3. 2015High-Dimensional Asymptotics of Prediction: Ridge Regression and Classificationcited by 148
  4. 2020The Gaussian equivalence of generative models for learning with shallow neural networkscited by 113
  5. 2020The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalizationcited by 101
  6. 2020Understanding Double Descent Requires a Fine-Grained Bias-Variance Decompositioncited by 76
  7. 2018A jamming transition from under- to over-parametrization affects loss landscape and generalizationcited by 67
  8. 2024Strong Model Collapsecited by 65
  9. 2020Asymptotics of Ridge (less) Regression under General Source Conditioncited by 62
  10. 2023High-dimensional analysis of double descent for linear regression with random projectionscited by 47
  11. 2023Demystifying Disagreement-on-the-Line in High Dimensionscited by 12
  12. 2021Fundamental tradeoffs between memorization and robustness in random features and neural tangent regimescited by 9
3 more
  1. 2015Applications of Realizations (aka Linearizations) to Free Probabilitycited by 8
  2. 2024An Effective Theory of Bias Amplificationcited by 5
  3. 2023Robust Linear Regression: Gradient-descent, Early-stopping, and Beyondcited by 4

Built on by

Not cited yet.