mlsanityData to 7 Oct 2026

X-SYS: A Reference Architecture for Interactive Explanation Systems

arXiv 2602.12748 ↗First posted 13 Feb 2026cs.AI cs.HC cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

The explainable AI (XAI) research community has proposed numerous technical methods, yet deploying explainability as systems remains challenging: Interactive explanation systems require both suitable algorithms and system capabilities that maintain explanation usability across repeated queries, evolving models and data, and governance constraints. We argue that operationalizing XAI requires treating explainability as an information systems problem where user interaction demands induce specific system requirements. We introduce X-SYS, a reference architecture for interactive explanation systems, that guides (X)AI researchers, developers and practitioners in connecting interactive explanation user interfaces (XUI) with system capabilities. X-SYS organizes around four quality attributes named STAR (scalability, traceability, responsiveness, and adaptability), and specifies a five-component decomposition (XUI Services, Explanation Services, Model Services, Data Services, Orchestration and Governance). It maps interaction patterns to system capabilities to decouple user interface evolution from backend computation. We implement X-SYS through SemanticLens, a system for semantic search and activation steering in vision-language models. SemanticLens demonstrates how contract-based service boundaries enable independent evolution, offline/online separation ensures responsiveness, and persistent state management supports traceability. Together, this work provides a reusable blueprint and concrete instantiation for interactive explanation systems supporting end-to-end design under operational constraints.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 37 of its 92 references that are in the map. First posted on arXiv 13 Feb 2026.

Builds on

37 of 92 references.

References · 37

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  6. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  7. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  8. 2023Steering Language Models With Activation Engineeringcited by 824
  9. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  10. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  11. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
  12. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 359
25 more
  1. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 343
  2. 2020Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Surveycited by 252
  3. 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 251
  4. 2019Explainable Machine Learning in Deploymentcited by 210
  5. 2014Qualitatively characterizing neural network optimization problemscited by 186
  6. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  7. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  8. 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
  9. 2020Principles and Practice of Explainable Machine Learningcited by 99
  10. 2023MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Trainingcited by 96
  11. 2020Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Studycited by 86
  12. 2021Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAycited by 53
  13. 2023Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAIcited by 33
  14. 2025Mechanistic understanding and validation of large AI models with SemanticLenscited by 29
  15. 2021Software Architecture for ML-based Systems: What Exists and What Lies Aheadcited by 13
  16. 2021How to choose an Explainability Method? Towards a Methodical Implementation of XAI in Practicecited by 9
  17. 2023Identifying Explanation Needs of End-users: Applying and Extending the XAI Question Bankcited by 8
  18. 2024Incremental XAI: Memorable Understanding of AI with Incremental Explanationscited by 8
  19. 2024Unpacking Human-AI interactions: From interaction primitives to a design spacecited by 7
  20. 2024Sparse, Efficient and Explainable Data Attribution with DualXDAcited by 6
  21. 2022Deep Learning Reproducibility and Explainable AI (XAI)cited by 4
  22. 2024Evaluating the Explainability of Attributes and Prototypes for a Medical Classification Modelcited by 4
  23. 2024What is the focus of XAI in UI design? Prioritizing UI design principles for enhancing XAI user experiencecited by 3
  24. 2024Prototype-Based Methods in Explainable AI and Emerging Opportunities in the Geosciencescited by 3
  25. 2024An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Servicescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Addressing the Selection Problem in Explainable AI
  2. 2026Explainable AI: Learning from the Learners
  3. 2022From Black Boxes to Conversations: Incorporating XAI in a Conversational Agent
  4. 2026Comparing Explanations is Not Enough, Explain the Change: New Standards are Needed to Explain Behavioral Shifts in Large Language Models
  5. 2026Explaining AI Agents Through Execution Traces
  6. 2025Explainable artificial intelligence (XAI): from inherent explainability to large language models
  7. 2026Towards Faithful Agentic XAI: A Verification Method and an Open-World Benchmark for Better Model Faithfulness
  8. 2026Editable XAI: Toward Bidirectional Human-AI Alignment with Co-Editable Explanations of Interpretable Attributes ↗