# Fundamentals of Regression

arXiv: https://arxiv.org/abs/2512.01920 · HTML: https://mlsanity.com/paper/2512.01920/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 80 references that are in the map.
- First posted on arXiv 27 Nov 2025.

## Builds on (most cited first, 16 of 16)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning](https://mlsanity.com/paper/2006.00719/) (2020)
- [Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances](https://mlsanity.com/paper/2105.12221/) (2021)
- [Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks](https://mlsanity.com/paper/2406.11045/) (2024)
- [Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusion](https://mlsanity.com/paper/2109.14860/) (2021)
- [Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs)](https://mlsanity.com/paper/2205.01059/) (2022)
- [Scalable Second Order Optimization for Deep Learning](https://mlsanity.com/paper/2002.09018/) (2020)
- [Lecture notes on ridge regression](https://mlsanity.com/paper/1509.09169/) (2015)
- [An adaptive augmented Lagrangian method for training physics and equality constrained artificial neural networks](https://mlsanity.com/paper/2306.04904/) (2023)
- [An overview and comparative analysis of Recurrent Neural Networks for Short Term Load Forecasting](https://mlsanity.com/paper/1705.04378/) (2017)
- [IDRLnet: A Physics-Informed Neural Network Library](https://mlsanity.com/paper/2107.04320/) (2021)
- [Maximum Likelihood Uncertainty Estimation: Robustness to Outliers](https://mlsanity.com/paper/2202.03870/) (2022)
- [Genetic Programming Based Symbolic Regression for Analytical Solutions to Differential Equations](https://mlsanity.com/paper/2302.03175/) (2023)

## Abstract (arXiv)

This chapter opens with a review of classic tools for regression, a subset of machine learning that seeks to find relationships between variables. With the advent of scientific machine learning this field has moved from a purely data-driven (statistical) formalism to a constrained or ``physics-informed'' formalism, which integrates physical knowledge and methods from traditional computational engineering. In the first part, we introduce the general concepts and the statistical flavor of regression versus other forms of curve fitting. We then move to an overview of traditional methods from machine learning and their classification and ways to link these to traditional computational science. Finally, we close with a note on methods to combine machine learning and numerical methods for physics