# Image classification network enhancement methods based on knowledge injection

arXiv: https://arxiv.org/abs/2401.04441 · HTML: https://mlsanity.com/paper/2401.04441/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 9 Jan 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Convolutional 2D Knowledge Graph Embeddings](https://mlsanity.com/paper/1707.01476/) (2017)
- [Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification](https://mlsanity.com/paper/2211.11158/) (2022)
- [Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems](https://mlsanity.com/paper/1903.12394/) (2019)
- [KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA](https://mlsanity.com/paper/2012.11014/) (2020)
- [Incorporating Domain Knowledge into Deep Neural Networks](https://mlsanity.com/paper/2103.00180/) (2021)
- [A Benchmark for Compositional Visual Reasoning](https://mlsanity.com/paper/2206.05379/) (2022)
- [RECON: Relation Extraction using Knowledge Graph Context in a Graph Neural Network](https://mlsanity.com/paper/2009.08694/) (2020)
- [Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning](https://mlsanity.com/paper/1912.00512/) (2019)
- [Knowledge Infused Decoding](https://mlsanity.com/paper/2204.03084/) (2022)
- [Identifying concept libraries from language about object structure](https://mlsanity.com/paper/2205.05666/) (2022)

## Abstract (arXiv)

The current deep neural network algorithm still stays in the end-to-end training supervision method like Image-Label pairs, which makes traditional algorithm is difficult to explain the reason for the results, and the prediction logic is difficult to understand and analyze. The current algorithm does not use the existing human knowledge information, which makes the model not in line with the human cognition model and makes the model not suitable for human use. In order to solve the above problems, the present invention provides a deep neural network training method based on the human knowledge, which uses the human cognition model to construct the deep neural network training model, and uses the existing human knowledge information to construct the deep neural network training model. This paper proposes a multi-level hierarchical deep learning algorithm, which is composed of multi-level hierarchical deep neural network architecture and multi-level hierarchical deep learning framework. The experimental results show that the proposed algorithm can effectively explain the hidden information of the neural network. The goal of our study is to improve the interpretability of deep neural networks (DNNs) by providing an analysis of the impact of knowledge injection on the classification task. We constructed a knowledge injection dataset with matching knowledge data and image classification data. The knowledge injection dataset is the benchmark dataset for the experiments in the paper. Our model expresses the improvement in interpretability and classification task performance of hidden layers at different scales.