# Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints

arXiv: https://arxiv.org/abs/2007.01380 · HTML: https://mlsanity.com/paper/2007.01380/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 18 of 18)

- [POMDP inference and robust solution via deep reinforcement learning: An application to railway optimal maintenance](https://mlsanity.com/paper/2307.08082/) (2023)
- [Safe Policy Improvement for POMDPs via Finite-State Controllers](https://mlsanity.com/paper/2301.04939/) (2023)
- [Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning](https://mlsanity.com/paper/2209.01092/) (2022)
- [Bridging POMDPs and Bayesian decision making for robust maintenance planning under model uncertainty: An application to railway systems](https://mlsanity.com/paper/2212.07933/) (2022)
- [IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARL](https://mlsanity.com/paper/2306.11551/) (2023)
- [Reinforcement and Deep Reinforcement Learning-based Solutions for Machine Maintenance Planning, Scheduling Policies, and Optimization](https://mlsanity.com/paper/2307.03860/) (2023)
- [Welfare Maximization Algorithm for Solving Budget-Constrained Multi-Component POMDPs](https://mlsanity.com/paper/2303.10302/) (2023)
- [Multi-agent deep reinforcement learning with centralized training and decentralized execution for transportation infrastructure management](https://mlsanity.com/paper/2401.12455/) (2024)
- [Policies for the Dynamic Traveling Maintainer Problem with Alerts](https://mlsanity.com/paper/2105.15119/) (2021)
- [Reinforcement Learning based Sequential Batch-sampling for Bayesian Optimal Experimental Design](https://mlsanity.com/paper/2112.10944/) (2021)
- [An investigation of belief-free DRL and MCTS for inspection and maintenance planning](https://mlsanity.com/paper/2312.14824/) (2023)
- [Deep Multi-Objective Reinforcement Learning for Utility-Based Infrastructural Maintenance Optimization](https://mlsanity.com/paper/2406.06184/) (2024)
- [A reinforcement learning agent for maintenance of deteriorating systems with increasingly imperfect repairs](https://mlsanity.com/paper/2505.20725/) (2025)
- [Hierarchical Deep Reinforcement Learning Framework for Multi-Year Asset Management Under Budget Constraints](https://mlsanity.com/paper/2507.19458/) (2025)
- [Interpretable Deep Reinforcement Learning for Element-level Bridge Life-cycle Optimization](https://mlsanity.com/paper/2604.02528/) (2026)
- [Smart Commander: A Hierarchical Reinforcement Learning Framework for Fleet-Level PHM Decision Optimization](https://mlsanity.com/paper/2604.07171/) (2026)
- [Planning or Learning: Reliability and Cost in Multi-Asset Maintenance](https://mlsanity.com/paper/2609.13566/) (2026)
- [Graph-Based Inference and Topology-Aware Multi-Agent Reinforcement Learning for Large-Scale Railway Network Management](https://mlsanity.com/paper/2609.30150/) (2026)

## Abstract (arXiv)

Determination of inspection and maintenance policies for minimizing long-term risks and costs in deteriorating engineering environments constitutes a complex optimization problem. Major computational challenges include the (i) curse of dimensionality, due to exponential scaling of state/action set cardinalities with the number of components; (ii) curse of history, related to exponentially growing decision-trees with the number of decision-steps; (iii) presence of state uncertainties, induced by inherent environment stochasticity and variability of inspection/monitoring measurements; (iv) presence of constraints, pertaining to stochastic long-term limitations, due to resource scarcity and other infeasible/undesirable system responses. In this work, these challenges are addressed within a joint framework of constrained Partially Observable Markov Decision Processes (POMDP) and multi-agent Deep Reinforcement Learning (DRL). POMDPs optimally tackle (ii)-(iii), combining stochastic dynamic programming with Bayesian inference principles. Multi-agent DRL addresses (i), through deep function parametrizations and decentralized control assumptions. Challenge (iv) is herein handled through proper state augmentation and Lagrangian relaxation, with emphasis on life-cycle risk-based constraints and budget limitations. The underlying algorithmic steps are provided, and the proposed framework is found to outperform well-established policy baselines and facilitate adept prescription of inspection and intervention actions, in cases where decisions must be made in the most resource- and risk-aware manner.