# AutoIP: A United Framework to Integrate Physics into Gaussian Processes

arXiv: https://arxiv.org/abs/2202.12316 · HTML: https://mlsanity.com/paper/2202.12316/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 46 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences](https://mlsanity.com/paper/1807.02582/) (2018)
- [Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems](https://mlsanity.com/paper/1809.08327/) (2018)
- [Machine Learning of Linear Differential Equations using Gaussian Processes](https://mlsanity.com/paper/1701.02440/) (2017)
- [Solving and Learning Nonlinear PDEs with Gaussian Processes](https://mlsanity.com/paper/2103.12959/) (2021)
- [Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks](https://mlsanity.com/paper/1905.01205/) (2019)
- [Learning unknown ODE models with Gaussian processes](https://mlsanity.com/paper/1803.04303/) (2018)
- [Physics-Informed Probabilistic Learning of Linear Embeddings of Non-linear Dynamics With Guaranteed Stability](https://mlsanity.com/paper/1906.03663/) (2019)
- [Determinantal Point Processes in Randomized Numerical Linear Algebra](https://mlsanity.com/paper/2005.03185/) (2020)
- [Multifidelity Modeling for Physics-Informed Neural Networks (PINNs)](https://mlsanity.com/paper/2106.13361/) (2021)
- [Fast Gaussian Process Based Gradient Matching for Parameter Identification in Systems of Nonlinear ODEs](https://mlsanity.com/paper/1804.04378/) (2018)
- [ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems](https://mlsanity.com/paper/1902.06278/) (2019)
- [Physics Informed Deep Kernel Learning](https://mlsanity.com/paper/2006.04976/) (2020)
- [Black-Box Inference for Non-Linear Latent Force Models](https://mlsanity.com/paper/1906.09199/) (2019)
- [Constraining the Dynamics of Deep Probabilistic Models](https://mlsanity.com/paper/1802.05680/) (2018)

## Built on by (most cited first, 20 of 20)

- [Error Analysis of Kernel/GP Methods for Nonlinear and Parametric PDEs](https://mlsanity.com/paper/2305.04962/) (2023)
- [A spectrum of physics-informed Gaussian processes for regression in engineering](https://mlsanity.com/paper/2309.10656/) (2023)
- [Solving High Frequency and Multi-Scale PDEs with Gaussian Processes](https://mlsanity.com/paper/2311.04465/) (2023)
- [Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology](https://mlsanity.com/paper/2310.05227/) (2023)
- [Random Grid Neural Processes for Parametric Partial Differential Equations](https://mlsanity.com/paper/2301.11040/) (2023)
- [Physics-Informed Variational State-Space Gaussian Processes](https://mlsanity.com/paper/2409.13876/) (2024)
- [Toward Efficient Kernel-Based Solvers for Nonlinear PDEs](https://mlsanity.com/paper/2410.11165/) (2024)
- [Equation Discovery with Bayesian Spike-and-Slab Priors and Efficient Kernels](https://mlsanity.com/paper/2310.05387/) (2023)
- [Gaussian Measures Conditioned on Nonlinear Observations: Consistency, MAP Estimators, and Simulation](https://mlsanity.com/paper/2405.13149/) (2024)
- [Position: Biology is the Challenge Physics-Informed ML Needs to Evolve](https://mlsanity.com/paper/2510.25368/) (2025)
- [Conditioning Gaussian Processes on Almost Anything](https://mlsanity.com/paper/2605.21041/) (2026)
- [Iterated INLA for State and Parameter Estimation in Nonlinear Dynamical Systems](https://mlsanity.com/paper/2402.17036/) (2024)
- [Understanding Generalization in Physics Informed Models through Affine Variety Dimensions](https://mlsanity.com/paper/2501.18879/) (2025)
- [LVM-GP: Uncertainty-Aware PDE Solver via coupling latent variable model and Gaussian process](https://mlsanity.com/paper/2507.22493/) (2025)
- [Tensor Gaussian Processes: Efficient Solvers for Nonlinear PDEs](https://mlsanity.com/paper/2510.13772/) (2025)
- [Domain Invariant Learning for Gaussian Processes and Bayesian Exploration](https://mlsanity.com/paper/2312.11318/) (2023)
- [Deep Gaussian Processes for Functional Maps](https://mlsanity.com/paper/2510.22068/) (2025)
- [Machine learning assisted state prediction of misspecified linear dynamical system via modal reduction](https://mlsanity.com/paper/2601.05297/) (2026)
- [Bayesian Nonparametric Mixed-Effect ODEs with Gaussian Processes](https://mlsanity.com/paper/2605.13088/) (2026)
- [Real-Time State-of-Health Estimation and Online Degradation Prognosis from Partial Battery Discharge Using Physics-Informed Neural Networks](https://mlsanity.com/paper/2608.14764/) (2026)

## Abstract (arXiv)

Physical modeling is critical for many modern science and engineering applications. From a data science or machine learning perspective, where more domain-agnostic, data-driven models are pervasive, physical knowledge -- often expressed as differential equations -- is valuable in that it is complementary to data, and it can potentially help overcome issues such as data sparsity, noise, and inaccuracy. In this work, we propose a simple, yet powerful and general framework -- AutoIP, for Automatically Incorporating Physics -- that can integrate all kinds of differential equations into Gaussian Processes (GPs) to enhance prediction accuracy and uncertainty quantification. These equations can be linear or nonlinear, spatial, temporal, or spatio-temporal, complete or incomplete with unknown source terms, and so on. Based on kernel differentiation, we construct a GP prior to sample the values of the target function, equation-related derivatives, and latent source functions, which are all jointly from a multivariate Gaussian distribution. The sampled values are fed to two likelihoods: one to fit the observations, and the other to conform to the equation. We use the whitening method to evade the strong dependency between the sampled function values and kernel parameters, and we develop a stochastic variational learning algorithm. AutoIP shows improvement upon vanilla GPs in both simulation and several real-world applications, even using rough, incomplete equations.