# Composable Building Blocks for Controllable and Transparent Interactive AI Systems

arXiv: https://arxiv.org/abs/2506.02262 · HTML: https://mlsanity.com/paper/2506.02262/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 33 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Builds on (most cited first, 13 of 13)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [ViperGPT: Visual Inference via Python Execution for Reasoning](https://mlsanity.com/paper/2303.08128/) (2023)
- [Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models](https://mlsanity.com/paper/2304.09842/) (2023)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [From Understanding to Utilization: A Survey on Explainability for Large Language Models](https://mlsanity.com/paper/2401.12874/) (2024)
- [Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computing](https://mlsanity.com/paper/2210.15889/) (2022)
- [Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant](https://mlsanity.com/paper/2501.17546/) (2025)
- [XAI for All: Can Large Language Models Simplify Explainable AI?](https://mlsanity.com/paper/2401.13110/) (2024)
- [Tell Me a Story! Narrative-Driven XAI with Large Language Models](https://mlsanity.com/paper/2309.17057/) (2023)
- [XAIR: A Framework of Explainable AI in Augmented Reality](https://mlsanity.com/paper/2303.16292/) (2023)
- [TimberTrek: Exploring and Curating Sparse Decision Trees with Interactive Visualization](https://mlsanity.com/paper/2209.09227/) (2022)

## Built on by (most cited first, 1 of 1)

- [MATCH: Engineering Transparent and Controllable Conversational XAI Systems through Composable Building Blocks](https://mlsanity.com/paper/2511.22420/) (2025)

## Abstract (arXiv)

While the increased integration of AI technologies into interactive systems enables them to solve an equally increasing number of tasks, the black box problem of AI models continues to spread throughout the interactive system as a whole. Explainable AI (XAI) techniques can make AI models more accessible by employing post-hoc methods or transitioning to inherently interpretable models. While this makes individual AI models clearer, the overarching system architecture remains opaque. To this end, we propose an approach to represent interactive systems as sequences of structural building blocks, such as AI models and control mechanisms grounded in the literature. These can then be explained through accompanying visual building blocks, such as XAI techniques. The flow and APIs of the structural building blocks form an explicit overview of the system. This serves as a communication basis for both humans and automated agents like LLMs, aligning human and machine interpretability of AI models. We discuss a selection of building blocks and concretize our flow-based approach in an architecture and accompanying prototype interactive system.