# Interpretable Hidden Markov Model-Based Deep Reinforcement Learning Hierarchical Framework for Predictive Maintenance of Turbofan Engines

arXiv: https://arxiv.org/abs/2206.13433 · HTML: https://mlsanity.com/paper/2206.13433/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 32 references that are in the map.
- First posted on arXiv 27 Jun 2022.

## Builds on (most cited first, 6 of 6)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning](https://mlsanity.com/paper/1811.00090/) (2018)
- [Interpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programming](https://mlsanity.com/paper/2106.11417/) (2021)
- [Predictive Maintenance for Edge-Based Sensor Networks: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2007.03313/) (2020)
- [Hidden Markov Model Estimation-Based Q-learning for Partially Observable Markov Decision Process](https://mlsanity.com/paper/1809.06401/) (2018)
- [Hidden Markov Models and their Application for Predicting Failure Events](https://mlsanity.com/paper/2005.09971/) (2020)

## Built on by (most cited first, 1 of 1)

- [Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces](https://mlsanity.com/paper/2310.14788/) (2023)

## Abstract (arXiv)

An open research question in deep reinforcement learning is how to focus the policy learning of key decisions within a sparse domain. This paper emphasizes combining the advantages of inputoutput hidden Markov models and reinforcement learning towards interpretable maintenance decisions. We propose a novel hierarchical-modeling methodology that, at a high level, detects and interprets the root cause of a failure as well as the health degradation of the turbofan engine, while, at a low level, it provides the optimal replacement policy. It outperforms the baseline performance of deep reinforcement learning methods applied directly to the raw data or when using a hidden Markov model without such a specialized hierarchy. It also provides comparable performance to prior work, however, with the additional benefit of interpretability.