# On the Soundness of XAI in Prognostics and Health Management (PHM)

arXiv: https://arxiv.org/abs/2303.05517 · HTML: https://mlsanity.com/paper/2303.05517/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 9 Mar 2023.

## Builds on (most cited first, 15 of 15)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Towards a Rigorous Evaluation of XAI Methods on Time Series](https://mlsanity.com/paper/1909.07082/) (2019)
- [Toward Explainable AI for Regression Models](https://mlsanity.com/paper/2112.11407/) (2021)
- [TSViz: Demystification of Deep Learning Models for Time-Series Analysis](https://mlsanity.com/paper/1802.02952/) (2018)
- [A stacked deep convolutional neural network to predict the remaining useful life of a turbofan engine](https://mlsanity.com/paper/2111.12689/) (2021)
- [Shedding Light on Black Box Machine Learning Algorithms: Development of an Axiomatic Framework to Assess the Quality of Methods that Explain Individual Predictions](https://mlsanity.com/paper/1808.05054/) (2018)

## Built on by (most cited first, 6 of 6)

- [Explainable Predictive Maintenance: A Survey of Current Methods, Challenges and Opportunities](https://mlsanity.com/paper/2401.07871/) (2024)
- [Interpretable Prognostics with Concept Bottleneck Models](https://mlsanity.com/paper/2405.17575/) (2024)
- [D3A-TS: Denoising-Driven Data Augmentation in Time Series](https://mlsanity.com/paper/2312.05550/) (2023)
- [Machine Learning Approaches for Diagnostics and Prognostics of Industrial Systems Using Open Source Data from PHM Data Challenges: A Review](https://mlsanity.com/paper/2312.16810/) (2023)
- [Towards transparent and data-driven fault detection in manufacturing: A case study on univariate, discrete time series](https://mlsanity.com/paper/2507.00102/) (2025)
- [A Heterogeneous Long-Micro Scale Cascading Architecture for General Aviation Health Management](https://mlsanity.com/paper/2603.22885/) (2026)

## Abstract (arXiv)

The aim of Predictive Maintenance, within the field of Prognostics and Health Management (PHM), is to identify and anticipate potential issues in the equipment before these become critical. The main challenge to be addressed is to assess the amount of time a piece of equipment will function effectively before it fails, which is known as Remaining Useful Life (RUL). Deep Learning (DL) models, such as Deep Convolutional Neural Networks (DCNN) and Long Short-Term Memory (LSTM) networks, have been widely adopted to address the task, with great success. However, it is well known that this kind of black box models are opaque decision systems, and it may be hard to explain its outputs to stakeholders (experts in the industrial equipment). Due to the large number of parameters that determine the behavior of these complex models, understanding the reasoning behind the predictions is challenging. This work presents a critical and comparative revision on a number of XAI methods applied on time series regression model for PM. The aim is to explore XAI methods within time series regression, which have been less studied than those for time series classification. The model used during the experimentation is a DCNN trained to predict the RUL of an aircraft engine. The methods are reviewed and compared using a set of metrics that quantifies a number of desirable properties that any XAI method should fulfill. The results show that GRAD-CAM is the most robust method, and that the best layer is not the bottom one, as is commonly seen within the context of Image Processing.