# Embedding Human Knowledge into Deep Neural Network via Attention Map

arXiv: https://arxiv.org/abs/1905.03540 · HTML: https://mlsanity.com/paper/1905.03540/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2019.

## Built on by (most cited first, 30 of 33)

- [MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiology](https://mlsanity.com/paper/2301.02228/) (2023)
- [Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images](https://mlsanity.com/paper/2302.14042/) (2023)
- [Leveraging Explanations in Interactive Machine Learning: An Overview](https://mlsanity.com/paper/2207.14526/) (2022)
- [Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvement](https://mlsanity.com/paper/2203.08008/) (2022)
- [Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning](https://mlsanity.com/paper/2212.03954/) (2022)
- [Towards Robust Classification Model by Counterfactual and Invariant Data Generation](https://mlsanity.com/paper/2106.01127/) (2021)
- [On Guiding Visual Attention with Language Specification](https://mlsanity.com/paper/2202.08926/) (2022)
- [Aligning Eyes between Humans and Deep Neural Network through Interactive Attention Alignment](https://mlsanity.com/paper/2202.02838/) (2022)
- [RES: A Robust Framework for Guiding Visual Explanation](https://mlsanity.com/paper/2206.13413/) (2022)
- [Studying How to Efficiently and Effectively Guide Models with Explanations](https://mlsanity.com/paper/2303.11932/) (2023)
- [UniBrain: Universal Brain MRI Diagnosis with Hierarchical Knowledge-enhanced Pre-training](https://mlsanity.com/paper/2309.06828/) (2023)
- [Human Attention-Guided Explainable Artificial Intelligence for Computer Vision Models](https://mlsanity.com/paper/2305.03601/) (2023)
- [Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales](https://mlsanity.com/paper/2404.03098/) (2024)
- [Reactive Model Correction: Mitigating Harm to Task-Relevant Features via Conditional Bias Suppression](https://mlsanity.com/paper/2404.09601/) (2024)
- [Improving the trustworthiness of image classification models by utilizing bounding-box annotations](https://mlsanity.com/paper/2108.10131/) (2021)
- [Interactive Acquisition of Fine-grained Visual Concepts by Exploiting Semantics of Generic Characterizations in Discourse](https://mlsanity.com/paper/2305.03461/) (2023)
- [Designing a Direct Feedback Loop between Humans and Convolutional Neural Networks through Local Explanations](https://mlsanity.com/paper/2307.04036/) (2023)
- [AIM: Amending Inherent Interpretability via Self-Supervised Masking](https://mlsanity.com/paper/2508.11502/) (2025)
- [Interpreting and improving deep-learning models with reality checks](https://mlsanity.com/paper/2108.06847/) (2021)
- [Interaction of a priori Anatomic Knowledge with Self-Supervised Contrastive Learning in Cardiac Magnetic Resonance Imaging](https://mlsanity.com/paper/2205.12429/) (2022)
- [XAI-Enhanced Semantic Segmentation Models for Visual Quality Inspection](https://mlsanity.com/paper/2401.09900/) (2024)
- [Wider Vision: Enriching Convolutional Neural Networks via Alignment to External Knowledge Bases](https://mlsanity.com/paper/2102.11132/) (2021)
- [SIM-ECG: A Signal Importance Mask-driven ECGClassification System](https://mlsanity.com/paper/2110.14835/) (2021)
- [Learning Robust Convolutional Neural Networks with Relevant Feature Focusing via Explanations](https://mlsanity.com/paper/2202.04237/) (2022)
- [Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognition](https://mlsanity.com/paper/2207.13306/) (2022)
- [Towards Better Guided Attention and Human Knowledge Insertion in Deep Convolutional Neural Networks](https://mlsanity.com/paper/2210.11177/) (2022)
- [Learning from AI: An Interactive Learning Method Using a DNN Model Incorporating Expert Knowledge as a Teacher](https://mlsanity.com/paper/2306.02257/) (2023)
- [Relevance-guided Neural Machine Translation](https://mlsanity.com/paper/2312.00214/) (2023)
- [3DPFIX: Improving Remote Novices' 3D Printing Troubleshooting through Human-AI Collaboration](https://mlsanity.com/paper/2401.15877/) (2024)
- [Towards a general framework for improving the performance of classifiers using XAI methods](https://mlsanity.com/paper/2403.10373/) (2024)

## Abstract (arXiv)

In this work, we aim to realize a method for embedding human knowledge into deep neural networks. While the conventional method to embed human knowledge has been applied for non-deep machine learning, it is challenging to apply it for deep learning models due to the enormous number of model parameters. To tackle this problem, we focus on the attention mechanism of an attention branch network (ABN). In this paper, we propose a fine-tuning method that utilizes a single-channel attention map which is manually edited by a human expert. Our fine-tuning method can train a network so that the output attention map corresponds to the edited ones. As a result, the fine-tuned network can output an attention map that takes into account human knowledge. Experimental results with ImageNet, CUB-200-2010, and IDRiD demonstrate that it is possible to obtain a clear attention map for a visual explanation and improve the classification performance. Our findings can be a novel framework for optimizing networks through human intuitive editing via a visual interface and suggest new possibilities for human-machine cooperation in addition to the improvement of visual explanations.