# Worth of knowledge in deep learning

arXiv: https://arxiv.org/abs/2307.00712 · HTML: https://mlsanity.com/paper/2307.00712/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 11 of 11)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems](https://mlsanity.com/paper/1903.12394/) (2019)
- [Robots that can adapt like animals](https://mlsanity.com/paper/1407.3501/) (2014)
- [Incorporating Symmetry into Deep Dynamics Models for Improved Generalization](https://mlsanity.com/paper/2002.03061/) (2020)
- [Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/1811.04026/) (2018)
- [Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study](https://mlsanity.com/paper/2001.03844/) (2020)
- [Deep Learning of Subsurface Flow via Theory-guided Neural Network](https://mlsanity.com/paper/1911.00103/) (2019)
- [Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method](https://mlsanity.com/paper/2012.06148/) (2020)
- [A Lagrangian Dual-based Theory-guided Deep Neural Network](https://mlsanity.com/paper/2008.10159/) (2020)
- [AutoKE: An automatic knowledge embedding framework for scientific machine learning](https://mlsanity.com/paper/2205.05390/) (2022)

## Abstract (arXiv)

Knowledge constitutes the accumulated understanding and experience that humans use to gain insight into the world. In deep learning, prior knowledge is essential for mitigating shortcomings of data-driven models, such as data dependence, generalization ability, and compliance with constraints. To enable efficient evaluation of the worth of knowledge, we present a framework inspired by interpretable machine learning. Through quantitative experiments, we assess the influence of data volume and estimation range on the worth of knowledge. Our findings elucidate the complex relationship between data and knowledge, including dependence, synergistic, and substitution effects. Our model-agnostic framework can be applied to a variety of common network architectures, providing a comprehensive understanding of the role of prior knowledge in deep learning models. It can also be used to improve the performance of informed machine learning, as well as distinguish improper prior knowledge.