# AutoKE: An automatic knowledge embedding framework for scientific machine learning

arXiv: https://arxiv.org/abs/2205.05390 · HTML: https://mlsanity.com/paper/2205.05390/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 56 references that are in the map.
- First posted on arXiv 11 May 2022.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning](https://mlsanity.com/paper/1907.04490/) (2019)
- [PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs](https://mlsanity.com/paper/1909.10145/) (2019)
- [Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS](https://mlsanity.com/paper/1912.06059/) (2019)
- [Physics-Informed Multi-LSTM Networks for Metamodeling of Nonlinear Structures](https://mlsanity.com/paper/2002.10253/) (2020)
- [DPM: A deep learning PDE augmentation method (with application to large-eddy simulation)](https://mlsanity.com/paper/1911.09145/) (2019)
- [Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)](https://mlsanity.com/paper/2106.11927/) (2021)
- [nPINNs: nonlocal Physics-Informed Neural Networks for a parametrized nonlocal universal Laplacian operator. Algorithms and Applications](https://mlsanity.com/paper/2004.04276/) (2020)
- [Deep Learning of Subsurface Flow via Theory-guided Neural Network](https://mlsanity.com/paper/1911.00103/) (2019)
- [Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method](https://mlsanity.com/paper/2012.06148/) (2020)
- [Efficient Uncertainty Quantification for Dynamic Subsurface Flow with Surrogate by Theory-guided Neural Network](https://mlsanity.com/paper/2004.13560/) (2020)
- [Integration of knowledge and data in machine learning](https://mlsanity.com/paper/2202.10337/) (2022)
- [Weak Form Theory-guided Neural Network (TgNN-wf) for Deep Learning of Subsurface Single and Two-phase Flow](https://mlsanity.com/paper/2009.04543/) (2020)
- [Theory-guided Auto-Encoder for Surrogate Construction and Inverse Modeling](https://mlsanity.com/paper/2011.08618/) (2020)
- [A Lagrangian Dual-based Theory-guided Deep Neural Network](https://mlsanity.com/paper/2008.10159/) (2020)
- [Deep-Learning Discovers Macroscopic Governing Equations for Viscous Gravity Currents from Microscopic Simulation Data](https://mlsanity.com/paper/2106.00009/) (2021)

## Built on by (most cited first, 6 of 6)

- [Physics-constrained robust learning of open-form partial differential equations from limited and noisy data](https://mlsanity.com/paper/2309.07672/) (2023)
- [Discovering Physics-Informed Neural Networks Model for Solving Partial Differential Equations through Evolutionary Computation](https://mlsanity.com/paper/2405.11208/) (2024)
- [DISCOVER: Deep identification of symbolically concise open-form PDEs via enhanced reinforcement-learning](https://mlsanity.com/paper/2210.02181/) (2022)
- [Filtered Partial Differential Equations: a robust surrogate constraint in physics-informed deep learning framework](https://mlsanity.com/paper/2311.03776/) (2023)
- [Implicit Neural Representation For Accurate CFD Flow Field Prediction](https://mlsanity.com/paper/2408.06486/) (2024)
- [Worth of knowledge in deep learning](https://mlsanity.com/paper/2307.00712/) (2023)

## Abstract (arXiv)

Imposing physical constraints on neural networks as a method of knowledge embedding has achieved great progress in solving physical problems described by governing equations. However, for many engineering problems, governing equations often have complex forms, including complex partial derivatives or stochastic physical fields, which results in significant inconveniences from the perspective of implementation. In this paper, a scientific machine learning framework, called AutoKE, is proposed, and a reservoir flow problem is taken as an instance to demonstrate that this framework can effectively automate the process of embedding physical knowledge. In AutoKE, an emulator comprised of deep neural networks (DNNs) is built for predicting the physical variables of interest. An arbitrarily complex equation can be parsed and automatically converted into a computational graph through the equation parser module, and the fitness of the emulator to the governing equation is evaluated via automatic differentiation. Furthermore, the fixed weights in the loss function are substituted with adaptive weights by incorporating the Lagrangian dual method. Neural architecture search (NAS) is also introduced into the AutoKE to select an optimal network architecture of the emulator according to the specific problem. Finally, we apply transfer learning to enhance the scalability of the emulator. In experiments, the framework is verified by a series of physical problems in which it can automatically embed physical knowledge into an emulator without heavy hand-coding. The results demonstrate that the emulator can not only make accurate predictions, but also be applied to similar problems with high efficiency via transfer learning.