mlsanityData to 7 Oct 2026

A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networks

arXiv 2107.10295 ↗First posted 21 Jul 2021cs.LG cs.AI cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a survey of ways in which existing scientific knowledge are included when constructing models with neural networks. The inclusion of domain-knowledge is of special interest not just to constructing scientific assistants, but also, many other areas that involve understanding data using human-machine collaboration. In many such instances, machine-based model construction may benefit significantly from being provided with human-knowledge of the domain encoded in a sufficiently precise form. This paper examines the inclusion of domain-knowledge by means of changes to: the input, the loss-function, and the architecture of deep networks. The categorisation is for ease of exposition: in practice we expect a combination of such changes will be employed. In each category, we describe techniques that have been shown to yield significant changes in the performance of deep neural networks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 51 of its 142 references that are in the map. First posted on arXiv 21 Jul 2021.

Builds on

51 of 142 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  6. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  7. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  8. 2014How transferable are features in deep neural networks?cited by 1,216
  9. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  10. 2016The Mythos of Model Interpretabilitycited by 716
  11. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  12. 2020Knowledge Graphscited by 619
38 more
  1. 2019Neural Probabilistic Logic Programming in DeepProbLogcited by 389
  2. 2017A Semantic Loss Function for Deep Learning with Symbolic Knowledgecited by 267
  3. 2017Learning Explanatory Rules from Noisy Datacited by 210
  4. 2015Simultaneous Deep Transfer Across Domains and Taskscited by 205
  5. 2019Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systemscited by 203
  6. 2017Neural-Symbolic Learning and Reasoning: A Survey and Interpretationcited by 195
  7. 2020DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learningcited by 159
  8. 2020Neural Unsupervised Domain Adaptation in NLP---A Surveycited by 142
  9. 2016Harnessing Deep Neural Networks with Logic Rulescited by 135
  10. 2020KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQAcited by 124
  11. 2019Knowledge Graph Convolutional Networks for Recommender Systemscited by 121
  12. 2018Embedding Logical Queries on Knowledge Graphscited by 120
  13. 2020Logical Neural Networkscited by 118
  14. 2019Towards Knowledge-Based Recommender Dialog Systemcited by 112
  15. 2019Artificial Intelligence: the global landscape of ethics guidelinescited by 107
  16. 2017Regularization for Deep Learning: A Taxonomycited by 91
  17. 2020From Statistical Relational to Neuro-Symbolic Artificial Intelligencecited by 63
  18. 2021The Modern Mathematics of Deep Learningcited by 63
  19. 2021DeepStochLog: Neural Stochastic Logic Programmingcited by 58
  20. 2019Augmenting Neural Networks with First-order Logiccited by 51
  21. 2019Embedding Symbolic Knowledge into Deep Networkscited by 49
  22. 2021Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?cited by 46
  23. 2019Estimating Node Importance in Knowledge Graphs Using Graph Neural Networkscited by 43
  24. 2020Turning 30: New Ideas in Inductive Logic Programmingcited by 42
  25. 2018Improving the Interpretability of Deep Neural Networks with Knowledge Distillationcited by 37
  26. 2020Semantics of the Black-Box: Can knowledge graphs help make deep learning systems more interpretable and explainable?cited by 31
  27. 2016Lifted Rule Injection for Relation Embeddingscited by 22
  28. 2019Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learningcited by 19
  29. 2020Incorporating Symbolic Domain Knowledge into Graph Neural Networkscited by 13
  30. 2020Knowledge Graphs and Knowledge Networks: The Story in Briefcited by 11
  31. 2018Logical Explanations for Deep Relational Machines Using Relevance Informationcited by 9
  32. 2020Beyond Graph Neural Networks with Lifted Relational Neural Networkscited by 9
  33. 2020Propositionalization and Embeddings: Two Sides of the Same Coincited by 8
  34. 2020"When they say weed causes depression, but it's your fav antidepressant": Knowledge-aware Attention Framework for Relationship Extractioncited by 8
  35. 2020Injecting Domain Knowledge in Neural Networks: a Controlled Experiment on a Constrained Problemcited by 7
  36. 2021"Who can help me?": Knowledge Infused Matching of Support Seekers and Support Providers during COVID-19 on Redditcited by 7
  37. 2021Inclusion of Domain-Knowledge into GNNs using Mode-Directed Inverse Entailmentcited by 6
  38. 2018Knowledge-Based Distant Regularization in Learning Probabilistic Modelscited by 3

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.