Risk-Sensitive and Robust Model-Based Reinforcement Learning and Planning
Not on the map. Why?
Abstract (arXiv)
Many sequential decision-making problems that are currently automated, such as those in manufacturing or recommender systems, operate in an environment where there is either little uncertainty, or zero risk of catastrophe. As companies and researchers attempt to deploy autonomous systems in less constrained environments, it is increasingly important that we endow sequential decision-making algorithms with the ability to reason about uncertainty and risk. In this thesis, we will address both planning and reinforcement learning (RL) approaches to sequential decision-making. In the planning setting, it is assumed that a model of the environment is provided, and a policy is optimised within that model. Reinforcement learning relies upon extensive random exploration, and therefore usually requires a simulator in which to perform training. In many real-world domains, it is impossible to construct a perfectly accurate model or simulator. Therefore, the performance of any policy is inevitably uncertain due to the incomplete knowledge about the environment. Furthermore, in stochastic domains, the outcome of any given run is also uncertain due to the inherent randomness of the environment. These two sources of uncertainty are usually classified as epistemic, and aleatoric uncertainty, respectively. The over-arching goal of this thesis is to contribute to developing algorithms that mitigate both sources of uncertainty in sequential decision-making problems. We make a number of contributions towards this goal, with a focus on model-based algorithms...
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 54 of its 186 references that are in the map. First posted on arXiv 2 Apr 2023.
Builds on
54 of 186 references.
References · 54
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
- 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
- 2016Concrete Problems in AI Safetycited by 1,582
- 2016Generative Adversarial Imitation Learningcited by 1,576
- 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
- 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
- 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
- 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
- 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
38 more
- 2019Behavior Regularized Offline Reinforcement Learningcited by 654
- 2020MOPO: Model-based Offline Policy Optimizationcited by 630
- 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
- 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
- 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 297
- 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
- 2018Model-Ensemble Trust-Region Policy Optimizationcited by 214
- 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
- 2019AlgaeDICE: Policy Gradient from Arbitrary Experiencecited by 187
- 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 167
- 2021Pessimistic Model-based Offline Reinforcement Learning under Partial Coveragecited by 160
- 2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 157
- 2021Offline RL Without Off-Policy Evaluationcited by 150
- 2020Deployment-Efficient Reinforcement Learning via Model-Based Offline Optimizationcited by 120
- 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
- 2022Adversarially Trained Actor Critic for Offline Reinforcement Learningcited by 111
- 2021Reward (Mis)design for Autonomous Drivingcited by 102
- 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 95
- 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
- 2017Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Predictioncited by 93
- 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 76
- 2022Supported Policy Optimization for Offline Reinforcement Learningcited by 73
- 2020A Game Theoretic Framework for Model Based Reinforcement Learningcited by 72
- 2013Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizationscited by 68
- 2021Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settingscited by 68
- 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 68
- 2021Risk-Averse Offline Reinforcement Learningcited by 65
- 2017Reinforcement Learning under Model Mismatchcited by 52
- 2019Worst Cases Policy Gradientscited by 52
- 2022Offline RL Policies Should be Trained to be Adaptivecited by 52
- 2021Offline Reinforcement Learning with Reverse Model-based Imaginationcited by 47
- 2019Off-Policy Policy Gradient with State Distribution Correctioncited by 43
- 2019An Optimistic Perspective on Offline Reinforcement Learningcited by 37
- 2019No-Regret Exploration in Goal-Oriented Reinforcement Learningcited by 34
- 2021Risk-Averse Bayes-Adaptive Reinforcement Learningcited by 28
- 2021Behavioral Priors and Dynamics Models: Improving Performance and Domain Transfer in Offline RLcited by 24
- 2022One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learningcited by 21
- 2017Batch Policy Gradient Methods for Improving Neural Conversation Modelscited by 10
50 most cited of 54 All 54 (JSON)
Built on by
Not cited yet.