# Generation of Multimodal Justification Using Visual Word Constraint Model for Explainable Computer-Aided Diagnosis

arXiv: https://arxiv.org/abs/1906.03922 · HTML: https://mlsanity.com/paper/1906.03922/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2019.

## Built on by (most cited first, 18 of 18)

- [Explainable artificial intelligence (XAI) in deep learning-based medical image analysis](https://mlsanity.com/paper/2107.10912/) (2021)
- [Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods](https://mlsanity.com/paper/2111.02398/) (2021)
- [Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond](https://mlsanity.com/paper/2102.01998/) (2021)
- [A Review on Explainability in Multimodal Deep Neural Nets](https://mlsanity.com/paper/2105.07878/) (2021)
- [Explainable Deep Learning Methods in Medical Image Classification: A Survey](https://mlsanity.com/paper/2205.04766/) (2022)
- [Explaining Chest X-ray Pathologies in Natural Language](https://mlsanity.com/paper/2207.04343/) (2022)
- [Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directions](https://mlsanity.com/paper/2306.05731/) (2023)
- [A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?](https://mlsanity.com/paper/2304.04780/) (2023)
- [Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks](https://mlsanity.com/paper/2410.02331/) (2024)
- [Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources](https://mlsanity.com/paper/2305.06217/) (2023)
- [A Framework for Interpretability in Machine Learning for Medical Imaging](https://mlsanity.com/paper/2310.01685/) (2023)
- [BI-RADS-Net: An Explainable Multitask Learning Approach for Cancer Diagnosis in Breast Ultrasound Images](https://mlsanity.com/paper/2110.04069/) (2021)
- [Deep Learning, Natural Language Processing, and Explainable Artificial Intelligence in the Biomedical Domain](https://mlsanity.com/paper/2202.12678/) (2022)
- [Interpretable Prediction of Pulmonary Hypertension in Newborns using Echocardiograms](https://mlsanity.com/paper/2203.13038/) (2022)
- [Post-Hoc Explainability of BI-RADS Descriptors in a Multi-task Framework for Breast Cancer Detection and Segmentation](https://mlsanity.com/paper/2308.14213/) (2023)
- [Evaluating Post-hoc Interpretability with Intrinsic Interpretability](https://mlsanity.com/paper/2305.03002/) (2023)
- [A Survey of Explainable AI and Proposal for a Discipline of Explanation Engineering](https://mlsanity.com/paper/2306.01750/) (2023)
- [Do We Trust What They Say or What They Do? A Multimodal User Embedding Provides Personalized Explanations](https://mlsanity.com/paper/2409.02965/) (2024)

## Abstract (arXiv)

The ambiguity of the decision-making process has been pointed out as the main obstacle to applying the deep learning-based method in a practical way in spite of its outstanding performance. Interpretability could guarantee the confidence of deep learning system, therefore it is particularly important in the medical field. In this study, a novel deep network is proposed to explain the diagnostic decision with visual pointing map and diagnostic sentence justifying result simultaneously. For the purpose of increasing the accuracy of sentence generation, a visual word constraint model is devised in training justification generator. To verify the proposed method, comparative experiments were conducted on the problem of the diagnosis of breast masses. Experimental results demonstrated that the proposed deep network could explain diagnosis more accurately with various textual justifications.