UAMDP: Uncertainty-Aware Markov Decision Process for Risk-Constrained Reinforcement Learning from Probabilistic Forecasts

arXiv 2510.08226 ↗First posted 9 Oct 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Sequential decisions in volatile, high-stakes settings require more than maximizing expected return; they require principled uncertainty management. This paper presents the Uncertainty-Aware Markov Decision Process (UAMDP), a unified framework that couples Bayesian forecasting, posterior-sampling reinforcement learning, and planning under a conditional value-at-risk (CVaR) constraint. In a closed loop, the agent updates its beliefs over latent dynamics, samples plausible futures via Thompson sampling, and optimizes policies subject to preset risk tolerances. We establish regret bounds that converge to the Bayes-optimal benchmark under standard regularity conditions. We evaluate UAMDP in two domains including high-frequency equity trading and retail inventory control, both marked by structural uncertainty and economic volatility. Relative to strong deep learning baselines, UAMDP improves long-horizon forecasting accuracy (RMSE decreases by up to 25% and sMAPE by 32%), and these gains translate into economic performance: the trading Sharpe ratio rises from 1.54 to 1.74 while maximum drawdown is roughly halved. These results show that integrating calibrated probabilistic modeling, exploration aligned with posterior uncertainty, and risk-aware control yields a robust, generalizable approach to safer and more profitable sequential decision-making.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 50 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

18 of 50 references.

References · 18

2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
2013Learning to Optimize Via Posterior Samplingcited by 256
2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
2017A Multi-Horizon Quantile Recurrent Forecastercited by 211
2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
2016Sequential Neural Models with Stochastic Layerscited by 115
2023Structured State Space Models for In-Context Reinforcement Learningcited by 111

6 more

2021Recent Advances in Reinforcement Learning in Financecited by 110
2014Optimizing the CVaR via Samplingcited by 99
2015Gated Feedback Recurrent Neural Networkscited by 54
2020Reinforcement Learning with Uncertainty Estimation for Tactical Decision-Making in Intersectionscited by 15
2020Bayesian Residual Policy Optimization: Scalable Bayesian Reinforcement Learning with Clairvoyant Expertscited by 4
2024On Risk-Sensitive Decision Making Under Uncertaintycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way