# Enriching Visual with Verbal Explanations for Relational Concepts -- Combining LIME with Aleph

arXiv: https://arxiv.org/abs/1910.01837 · HTML: https://mlsanity.com/paper/1910.01837/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Oct 2019.

## Built on by (most cited first, 23 of 23)

- [Pix2Code: Learning to Compose Neural Visual Concepts as Programs](https://mlsanity.com/paper/2402.08280/) (2024)
- [Learning by Self-Explaining](https://mlsanity.com/paper/2309.08395/) (2023)
- [SurvNAM: The machine learning survival model explanation](https://mlsanity.com/paper/2104.08903/) (2021)
- [Explainable Machine Learning with Prior Knowledge: An Overview](https://mlsanity.com/paper/2105.10172/) (2021)
- [Expressive Explanations of DNNs by Combining Concept Analysis with ILP](https://mlsanity.com/paper/2105.07371/) (2021)
- [TRUST XAI: Model-Agnostic Explanations for AI With a Case Study on IIoT Security](https://mlsanity.com/paper/2205.01232/) (2022)
- [Ensembles of Random SHAPs](https://mlsanity.com/paper/2103.03302/) (2021)
- [Which LIME should I trust? Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2503.24365/) (2025)
- [An Imprecise SHAP as a Tool for Explaining the Class Probability Distributions under Limited Training Data](https://mlsanity.com/paper/2106.09111/) (2021)
- [Attention-like feature explanation for tabular data](https://mlsanity.com/paper/2108.04855/) (2021)
- [SurvBeX: An explanation method of the machine learning survival models based on the Beran estimator](https://mlsanity.com/paper/2308.03730/) (2023)
- [LIMEcraft: Handcrafted superpixel selection and inspection for Visual eXplanations](https://mlsanity.com/paper/2111.08094/) (2021)
- [An Interactive Explanatory AI System for Industrial Quality Control](https://mlsanity.com/paper/2203.09181/) (2022)
- [Interpretable Ensembles of Hyper-Rectangles as Base Models](https://mlsanity.com/paper/2303.08625/) (2023)
- [Position: Do Not Explain Vision Models Without Context](https://mlsanity.com/paper/2404.18316/) (2024)
- [Enriching Artificial Intelligence Explanations with Knowledge Fragments](https://mlsanity.com/paper/2204.05579/) (2022)
- [When a Relation Tells More Than a Concept: Exploring and Evaluating Classifier Decisions with CoReX](https://mlsanity.com/paper/2405.01661/) (2024)
- [Towards Learning a Vocabulary of Visual Concepts and Operators using Deep Neural Networks](https://mlsanity.com/paper/2109.00479/) (2021)
- [SurvBeNIM: The Beran-Based Neural Importance Model for Explaining the Survival Models](https://mlsanity.com/paper/2312.06638/) (2023)
- [Dual feature-based and example-based explanation methods](https://mlsanity.com/paper/2401.16294/) (2024)
- [Enhancing Deep Learning Model Explainability in Brain Tumor Datasets using Post-Heuristic Approaches](https://mlsanity.com/paper/2404.19568/) (2024)
- [Explaining Neural Networks in Preference Learning: a Post-hoc Inductive Logic Programming Approach](https://mlsanity.com/paper/2604.06838/) (2026)
- [HyDI: A hybrid Deep Learning-Inductive Logic Programming ensemble for multi-label classification](https://mlsanity.com/paper/2609.37740/) (2026)

## Abstract (arXiv)

With the increasing number of deep learning applications, there is a growing demand for explanations. Visual explanations provide information about which parts of an image are relevant for a classifier's decision. However, highlighting of image parts (e.g., an eye) cannot capture the relevance of a specific feature value for a class (e.g., that the eye is wide open). Furthermore, highlighting cannot convey whether the classification depends on the mere presence of parts or on a specific spatial relation between them. Consequently, we present an approach that is capable of explaining a classifier's decision in terms of logic rules obtained by the Inductive Logic Programming system Aleph. The examples and the background knowledge needed for Aleph are based on the explanation generation method LIME. We demonstrate our approach with images of a blocksworld domain. First, we show that our approach is capable of identifying a single relation as important explanatory construct. Afterwards, we present the more complex relational concept of towers. Finally, we show how the generated relational rules can be explicitly related with the input image, resulting in richer explanations.