Upside-Down Reinforcement Learning for More Interpretable Optimal Control
On the map: Deep reinforcement learning methods → Deep RL algorithms and representations
Abstract (arXiv)
Model-Free Reinforcement Learning (RL) algorithms either learn how to map states to expected rewards or search for policies that can maximize a certain performance function. Model-Based algorithms instead, aim to learn an approximation of the underlying model of the RL environment and then use it in combination with planning algorithms. Upside-Down Reinforcement Learning (UDRL) is a novel learning paradigm that aims to learn how to predict actions from states and desired commands. This task is formulated as a Supervised Learning problem and has successfully been tackled by Neural Networks (NNs). In this paper, we investigate whether function approximation algorithms other than NNs can also be used within a UDRL framework. Our experiments, performed over several popular optimal control benchmarks, show that tree-based methods like Random Forests and Extremely Randomized Trees can perform just as well as NNs with the significant benefit of resulting in policies that are inherently more interpretable than NNs, therefore paving the way for more transparent, safe, and robust RL.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 20 of its 49 references that are in the map. First posted on arXiv 18 Nov 2024.
Builds on
20 of 49 references.
References · 20
- 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
- 2021Tabular Data: Deep Learning is Not All You Needcited by 535
- 2019Reinforcement Learning in Healthcare: A Surveycited by 310
- 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
- 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 225
- 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
- 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 125
- 2019Training Agents using Upside-Down Reinforcement Learningcited by 107
- 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 106
- 2021A Survey on Interpretable Reinforcement Learningcited by 73
- 2017A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Unitscited by 50
- 2024shapiq: Shapley Interactions for Machine Learningcited by 43
8 more
- 2018Q-learning with Nearest Neighborscited by 37
- 2019Global Explanations of Neural Networks: Mapping the Landscape of Predictionscited by 32
- 2024A comprehensive study on fidelity metrics for XAIcited by 16
- 2021A Boosting Approach to Reinforcement Learningcited by 12
- 2023IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARLcited by 9
- 2024Decision Predicate Graphs: Enhancing Interpretability in Tree Ensemblescited by 5
- 2022Learning Relative Return Policies With Upside-Down Reinforcement Learningcited by 3
- 2022Q-learning with online random forestscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Upside Down Reinforcement Learning with Policy Generators
- 2022All You Need Is Supervised Learning: From Imitation Learning to Meta-RL With Upside Down RL
- 2025On the Convergence and Stability of Upside-Down Reinforcement Learning, Goal-Conditioned Supervised Learning, and Online Decision Transformers
- 2024Mitigating Information Loss in Tree-Based Reinforcement Learning via Direct Optimization
- 2025TreeRL: LLM Reinforcement Learning with On-Policy Tree Search
- 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning
- 2022A Unified Framework for Alternating Offline Model Training and Policy Learning
- 2020MOReL : Model-Based Offline Reinforcement Learning