# A User Study on Explainable Online Reinforcement Learning for Adaptive Systems

arXiv: https://arxiv.org/abs/2307.04098 · HTML: https://mlsanity.com/paper/2307.04098/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 83 references that are in the map.
- First posted on arXiv 9 Jul 2023.

## Builds on (most cited first, 26 of 26)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Bridging the Gap Between Value and Policy Based Reinforcement Learning](https://mlsanity.com/paper/1702.08892/) (2017)
- [RUDDER: Return Decomposition for Delayed Rewards](https://mlsanity.com/paper/1806.07857/) (2018)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [Towards Explainable Artificial Intelligence](https://mlsanity.com/paper/1909.12072/) (2019)
- [Explainability in Deep Reinforcement Learning](https://mlsanity.com/paper/2008.06693/) (2020)
- [Interestingness Elements for Explainable Reinforcement Learning: Understanding Agents' Capabilities and Limitations](https://mlsanity.com/paper/1912.09007/) (2019)
- [GLocalX -- From Local to Global Explanations of Black Box AI Models](https://mlsanity.com/paper/2101.07685/) (2021)
- [The Challenge of Crafting Intelligible Intelligence](https://mlsanity.com/paper/1803.04263/) (2018)
- [A systematic review and taxonomy of explanations in decision support and recommender systems](https://mlsanity.com/paper/2006.08672/) (2020)
- [A Survey and Taxonomy of Self-Aware and Self-Adaptive Cloud Autoscaling Systems](https://mlsanity.com/paper/1609.03590/) (2016)
- [Perpetual Assurances for Self-Adaptive Systems](https://mlsanity.com/paper/1903.04771/) (2019)
- [Feature-Model-Guided Online Learning for Self-Adaptive Systems](https://mlsanity.com/paper/1907.09158/) (2019)
- [Evolution of statistical analysis in empirical software engineering research: Current state and steps forward](https://mlsanity.com/paper/1706.00933/) (2017)
- [Decentralized Self-Adaptive Systems: A Mapping Study](https://mlsanity.com/paper/2103.09074/) (2021)
- [Explaining Online Reinforcement Learning Decisions of Self-Adaptive Systems](https://mlsanity.com/paper/2210.05931/) (2022)
- [ReCCoVER: Detecting Causal Confusion for Explainable Reinforcement Learning](https://mlsanity.com/paper/2203.11211/) (2022)
- [Self-Adaptation in Industry: A Survey](https://mlsanity.com/paper/2211.03116/) (2022)
- [Preliminary Results of a Survey on the Use of Self-Adaptation in Industry](https://mlsanity.com/paper/2204.06816/) (2022)
- [A Meta Reinforcement Learning-based Approach for Self-Adaptive System](https://mlsanity.com/paper/2105.04986/) (2021)
- [Self-adaptive Architectures in IoT Systems: A Systematic Literature Review](https://mlsanity.com/paper/2109.03312/) (2021)

## Built on by (most cited first, 3 of 3)

- [XABPs: Towards eXplainable Autonomous Business Processes](https://mlsanity.com/paper/2507.23269/) (2025)
- [An AI Chatbot for Explaining Deep Reinforcement Learning Decisions of Service-oriented Systems](https://mlsanity.com/paper/2309.14391/) (2023)
- [Explaining Reinforcement Learning Decisions in Self-adaptive Systems](https://mlsanity.com/paper/2608.14620/) (2026)

## Abstract (arXiv)

Online reinforcement learning (RL) is increasingly used for realizing adaptive systems in the presence of design time uncertainty. Online RL facilitates learning from actual operational data and thereby leverages feedback only available at runtime. However, Online RL requires the definition of an effective and correct reward function, which quantifies the feedback to the RL algorithm and thereby guides learning. With Deep RL gaining interest, the learned knowledge is no longer explicitly represented, but is represented as a neural network. For a human, it becomes practically impossible to relate the parametrization of the neural network to concrete RL decisions. Deep RL thus essentially appears as a black box, which severely limits the debugging of adaptive systems. We previously introduced the explainable RL technique XRL-DINE, which provides visual insights into why certain decisions were made at important time points. Here, we introduce an empirical user study involving 54 software engineers from academia and industry to assess (1) the performance of software engineers when performing different tasks using XRL-DINE and (2) the perceived usefulness and ease of use of XRL-DINE.