# Open the Black Box Data-Driven Explanation of Black Box Decision Systems

arXiv: https://arxiv.org/abs/1806.09936 · HTML: https://mlsanity.com/paper/1806.09936/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2018.

## Built on by (most cited first, 9 of 9)

- [How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review](https://mlsanity.com/paper/2107.12045/) (2021)
- [CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models](https://mlsanity.com/paper/2109.01401/) (2021)
- [Human-Centric Artificial Intelligence Architecture for Industry 5.0 Applications](https://mlsanity.com/paper/2203.10794/) (2022)
- [Auditing of AI: Legal, Ethical and Technical Approaches](https://mlsanity.com/paper/2407.06235/) (2024)
- [Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare](https://mlsanity.com/paper/2211.16444/) (2022)
- [XAI-KG: knowledge graph to support XAI and decision-making in manufacturing](https://mlsanity.com/paper/2105.01929/) (2021)
- [Semantic XAI for contextualized demand forecasting explanations](https://mlsanity.com/paper/2104.00452/) (2021)
- [Enriching Artificial Intelligence Explanations with Knowledge Fragments](https://mlsanity.com/paper/2204.05579/) (2022)
- [Interpretable Outlier Summarization](https://mlsanity.com/paper/2303.06261/) (2023)

## Abstract (arXiv)

Black box systems for automated decision making, often based on machine learning over (big) data, map a user's features into a class or a score without exposing the reasons why. This is problematic not only for lack of transparency, but also for possible biases hidden in the algorithms, due to human prejudices and collection artifacts hidden in the training data, which may lead to unfair or wrong decisions. We introduce the local-to-global framework for black box explanation, a novel approach with promising early results, which paves the road for a wide spectrum of future developments along three dimensions: (i) the language for expressing explanations in terms of highly expressive logic-based rules, with a statistical and causal interpretation; (ii) the inference of local explanations aimed at revealing the logic of the decision adopted for a specific instance by querying and auditing the black box in the vicinity of the target instance; (iii), the bottom-up generalization of the many local explanations into simple global ones, with algorithms that optimize the quality and comprehensibility of explanations.