mlsanityData to 7 Oct 2026

Bridging POMDPs and Bayesian decision making for robust maintenance planning under model uncertainty: An application to railway systems

arXiv 2212.07933 ↗First posted 15 Dec 2022cs.AI cs.LG cs.SYcited by 11

Not on the map. Why?

Abstract (arXiv)

Structural Health Monitoring (SHM) describes a process for inferring quantifiable metrics of structural condition, which can serve as input to support decisions on the operation and maintenance of infrastructure assets. Given the long lifespan of critical structures, this problem can be cast as a sequential decision making problem over prescribed horizons. Partially Observable Markov Decision Processes (POMDPs) offer a formal framework to solve the underlying optimal planning task. However, two issues can undermine the POMDP solutions. Firstly, the need for a model that can adequately describe the evolution of the structural condition under deterioration or corrective actions and, secondly, the non-trivial task of recovery of the observation process parameters from available monitoring data. Despite these potential challenges, the adopted POMDP models do not typically account for uncertainty on model parameters, leading to solutions which can be unrealistically confident. In this work, we address both key issues. We present a framework to estimate POMDP transition and observation model parameters directly from available data, via Markov Chain Monte Carlo (MCMC) sampling of a Hidden Markov Model (HMM) conditioned on actions. The MCMC inference estimates distributions of the involved model parameters. We then form and solve the POMDP problem by exploiting the inferred distributions, to derive solutions that are robust to model uncertainty. We successfully apply our approach on maintenance planning for railway track assets on the basis of a "fractal value" indicator, which is computed from actual railway monitoring data.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 10 of its 62 references that are in the map. First posted on arXiv 15 Dec 2022.

Builds on

10 of 62 references.

References · 10

  1. 2018On Improving Deep Reinforcement Learning for POMDPscited by 49
  2. 2018Managing engineering systems with large state and action spaces through deep reinforcement learningcited by 32
  3. 2020Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraintscited by 18
  4. 2020An On-Line POMDP Solver for Continuous Observation Spacescited by 15
  5. 2021On risk-based active learning for structural health monitoringcited by 15
  6. 2020Optimal Inspection and Maintenance Planning for Deteriorating Structural Components through Dynamic Bayesian Networks and Markov Decision Processescited by 12
  7. 2022Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learningcited by 11
  8. 2021Value of information from vibration-based structural health monitoring extracted via Bayesian model updatingcited by 10
  9. 2021Which Model to Trust: Assessing the Influence of Models on the Performance of Reinforcement Learning Algorithms for Continuous Control Taskscited by 3
  10. 2021Autoregressive Hidden Markov Models with partial knowledge on latent space applied to aero-engines prognosticscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 4; 2026: 2.

11 papers.

  1. 2023POMDP inference and robust solution via deep reinforcement learning: An application to railway optimal maintenancecited by 13
  2. 2023A digital twin framework for civil engineering structurescited by 10
  3. 2024Multi-agent deep reinforcement learning with centralized training and decentralized execution for transportation infrastructure managementcited by 5
  4. 2023Quantifying the value of information transfer in population-based SHMcited by 3
  5. 2024A local squared Wasserstein-2 method for efficient reconstruction of models with uncertaintycited by 2
  6. 2025Deep Belief Markov Models for POMDP Inferencecited by 2
  7. 2025A generative adversarial network optimization method for damage detection and digital twinning by deep AI fault learning: Z24 Bridge structural health monitoring benchmark validationcited by 1
  8. 2025Real-Time Structural Health Monitoring with Bayesian Neural Networks: Distinguishing Aleatoric and Epistemic Uncertainty for Digital Twin Frameworkscited by 0
  9. 2025Adaptive digital twins for predictive decision-making: Online Bayesian learning of transition dynamicscited by 0
  10. 2026Optimal sequential decision-making for error propagation mitigation in digital twinscited by 0
  11. 2026Graph-Based Inference and Topology-Aware Multi-Agent Reinforcement Learning for Large-Scale Railway Network Managementcited by 0