# Towards an MLOps Architecture for XAI in Industrial Applications

arXiv: https://arxiv.org/abs/2309.12756 · HTML: https://mlsanity.com/paper/2309.12756/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 74 references that are in the map.
- First posted on arXiv 22 Sep 2023.

## Builds on (most cited first, 12 of 12)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Making deep neural networks right for the right scientific reasons by interacting with their explanations](https://mlsanity.com/paper/2001.05371/) (2020)
- [Explaining Explanations in AI](https://mlsanity.com/paper/1811.01439/) (2018)
- [MLOps -- Definitions, Tools and Challenges](https://mlsanity.com/paper/2201.00162/) (2022)
- [Concept Drift Detection for Streaming Data](https://mlsanity.com/paper/1504.01044/) (2015)
- [timeXplain -- A Framework for Explaining the Predictions of Time Series Classifiers](https://mlsanity.com/paper/2007.07606/) (2020)
- [Concept Drift Detection and Adaptation with Hierarchical Hypothesis Testing](https://mlsanity.com/paper/1707.07821/) (2017)
- [Explainable Failure Predictions with RNN Classifiers based on Time Series Data](https://mlsanity.com/paper/1901.08554/) (2019)
- [Test Automation with Grad-CAM Heatmaps -- A Future Pipe Segment in MLOps for Vision AI?](https://mlsanity.com/paper/2103.01837/) (2021)

## Built on by (most cited first, 2 of 2)

- [Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda](https://mlsanity.com/paper/2404.09554/) (2024)
- [A Continuous-Time Consistency Model for 3D Point Cloud Generation](https://mlsanity.com/paper/2509.01492/) (2025)

## Abstract (arXiv)

Machine learning (ML) has become a popular tool in the industrial sector as it helps to improve operations, increase efficiency, and reduce costs. However, deploying and managing ML models in production environments can be complex. This is where Machine Learning Operations (MLOps) comes in. MLOps aims to streamline this deployment and management process. One of the remaining MLOps challenges is the need for explanations. These explanations are essential for understanding how ML models reason, which is key to trust and acceptance. Better identification of errors and improved model accuracy are only two resulting advantages. An often neglected fact is that deployed models are bypassed in practice when accuracy and especially explainability do not meet user expectations. We developed a novel MLOps software architecture to address the challenge of integrating explanations and feedback capabilities into the ML development and deployment processes. In the project EXPLAIN, our architecture is implemented in a series of industrial use cases. The proposed MLOps software architecture has several advantages. It provides an efficient way to manage ML models in production environments. Further, it allows for integrating explanations into the development and deployment processes.