# Integration of knowledge and data in machine learning

arXiv: https://arxiv.org/abs/2202.10337 · HTML: https://mlsanity.com/paper/2202.10337/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 74 references that are in the map.
- First posted on arXiv 15 Feb 2022.

## Builds on (most cited first, 25 of 25)

- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network](https://mlsanity.com/paper/1812.04426/) (2018)
- [PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain](https://mlsanity.com/paper/2004.13145/) (2020)
- [Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data](https://mlsanity.com/paper/1612.08544/) (2016)
- [Physics-informed learning of governing equations from scarce data](https://mlsanity.com/paper/2005.03448/) (2020)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling](https://mlsanity.com/paper/1710.11431/) (2017)
- [Constrained Convolutional Neural Networks for Weakly Supervised Segmentation](https://mlsanity.com/paper/1506.03648/) (2015)
- [Reinforcement Learning of Theorem Proving](https://mlsanity.com/paper/1805.07563/) (2018)
- [DLGA-PDE: Discovery of PDEs with incomplete candidate library via combination of deep learning and genetic algorithm](https://mlsanity.com/paper/2001.07305/) (2020)
- [Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)](https://mlsanity.com/paper/2106.11927/) (2021)
- [Deep Learning of Subsurface Flow via Theory-guided Neural Network](https://mlsanity.com/paper/1911.00103/) (2019)
- [Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method](https://mlsanity.com/paper/2012.06148/) (2020)
- [Discovering Nonlinear PDEs from Scarce Data with Physics-encoded Learning](https://mlsanity.com/paper/2201.12354/) (2022)
- [Robust discovery of partial differential equations in complex situations](https://mlsanity.com/paper/2106.00008/) (2021)
- [Deep-learning of Parametric Partial Differential Equations from Sparse and Noisy Data](https://mlsanity.com/paper/2005.07916/) (2020)
- [Efficient Uncertainty Quantification for Dynamic Subsurface Flow with Surrogate by Theory-guided Neural Network](https://mlsanity.com/paper/2004.13560/) (2020)
- [Deep-learning based discovery of partial differential equations in integral form from sparse and noisy data](https://mlsanity.com/paper/2011.11981/) (2020)
- [Identification of physical processes via combined data-driven and data-assimilation methods](https://mlsanity.com/paper/1810.11977/) (2018)
- [Saliency Preservation in Low-Resolution Grayscale Images](https://mlsanity.com/paper/1712.02048/) (2017)
- [Discovering PDEs from Multiple Experiments](https://mlsanity.com/paper/2109.11939/) (2021)

## Built on by (most cited first, 13 of 13)

- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [Discrete Point-wise Attack Is Not Enough: Generalized Manifold Adversarial Attack for Face Recognition](https://mlsanity.com/paper/2301.06083/) (2022)
- [LLM4ED: Large Language Models for Automatic Equation Discovery](https://mlsanity.com/paper/2405.07761/) (2024)
- [AutoKE: An automatic knowledge embedding framework for scientific machine learning](https://mlsanity.com/paper/2205.05390/) (2022)
- [DISCOVER: Deep identification of symbolically concise open-form PDEs via enhanced reinforcement-learning](https://mlsanity.com/paper/2210.02181/) (2022)
- [Depression Detection Using Digital Traces on Social Media: A Knowledge-aware Deep Learning Approach](https://mlsanity.com/paper/2303.05389/) (2023)
- [A Data-Driven Framework for Discovering Fractional Differential Equations in Complex Systems](https://mlsanity.com/paper/2412.03970/) (2024)
- [TgDLF2.0: Theory-guided deep-learning for electrical load forecasting via Transformer and transfer learning](https://mlsanity.com/paper/2210.02448/) (2022)
- [Discovering an interpretable mathematical expression for a full wind-turbine wake with artificial intelligence enhanced symbolic regression](https://mlsanity.com/paper/2406.00695/) (2024)
- [Knowledge-Injected Federated Learning](https://mlsanity.com/paper/2208.07530/) (2022)
- [Filtered Partial Differential Equations: a robust surrogate constraint in physics-informed deep learning framework](https://mlsanity.com/paper/2311.03776/) (2023)
- [Transforming organic chemistry research paradigms: moving from manual efforts to the intersection of automation and artificial intelligence](https://mlsanity.com/paper/2312.00808/) (2023)
- [A Noise-robust Multi-head Attention Mechanism for Formation Resistivity Prediction: Frequency Aware LSTM](https://mlsanity.com/paper/2406.03849/) (2024)

## Abstract (arXiv)

Scientific research's mandate is to comprehend and explore the world, as well as to improve it based on experience and knowledge. Knowledge embedding and knowledge discovery are two significant methods of integrating knowledge and data. Through knowledge embedding, the barriers between knowledge and data can be eliminated, and machine learning models with physical common sense can be established. Meanwhile, humans' understanding of the world is always limited, and knowledge discovery takes advantage of machine learning to extract new knowledge from observations. Knowledge discovery can not only assist researchers to better grasp the nature of physics, but it can also support them in conducting knowledge embedding research. A closed loop of knowledge generation and usage are formed by combining knowledge embedding with knowledge discovery, which can improve the robustness and accuracy of models and uncover previously unknown scientific principles. This study summarizes and analyzes extant literature, as well as identifies research gaps and future opportunities.