mlsanityData to 7 Oct 2026

Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examples

arXiv 2310.07747 ↗First posted 11 Oct 2023cs.LG cs.AI cs.ROcited by 9

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Learning controllers with offline data in decision-making systems is an essential area of research due to its potential to reduce the risk of applications in real-world systems. However, in responsibility-sensitive settings such as healthcare, decision accountability is of paramount importance, yet has not been adequately addressed by the literature. This paper introduces the Accountable Offline Controller (AOC) that employs the offline dataset as the Decision Corpus and performs accountable control based on a tailored selection of examples, referred to as the Corpus Subset. AOC operates effectively in low-data scenarios, can be extended to the strictly offline imitation setting, and displays qualities of both conservation and adaptability. We assess AOC's performance in both simulated and real-world healthcare scenarios, emphasizing its capability to manage offline control tasks with high levels of performance while maintaining accountability.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 61 of its 94 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

61 of 94 references.

Closest ancestors · 12

Same direction, newest first

  1. 2023Neural Laplace Control for Continuous-time Delayed Systems24 Feb
  2. 2023Anti-Exploration by Random Network Distillation31 Jan
  3. 2022RORL: Robust Offline Reinforcement Learning via Conservative Smoothing6 Jun
  4. 2022Adversarially Trained Actor Critic for Offline Reinforcement Learning5 Feb
  5. 2021Offline Reinforcement Learning with Value-based Episodic Memory19 Oct
  6. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble4 Oct
  7. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  8. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  9. 2020MOPO: Model-based Offline Policy Optimization27 May
  10. 2020MOReL : Model-Based Offline Reinforcement Learning12 May
  11. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  12. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 49

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2016OpenAI Gymcited by 1,684
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  8. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  9. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  10. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  11. 2018Programmatically Interpretable Reinforcement Learningcited by 192
  12. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
37 more
  1. 2017Neural Episodic Controlcited by 148
  2. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  3. 2016Model-Free Episodic Controlcited by 100
  4. 2020BADGR: An Autonomous Self-Supervised Learning-Based Navigation Systemcited by 97
  5. 2020Human-centered Explainable AI: Towards a Reflective Sociotechnical Approachcited by 90
  6. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  7. 2019Towards Interpretable Reinforcement Learning Using Attention Augmented Agentscited by 82
  8. 2021A Survey on Interpretable Reinforcement Learningcited by 70
  9. 2018Toward Interpretable Deep Reinforcement Learning with Linear Model U-Treescited by 65
  10. 2019Meta-Q-Learningcited by 62
  11. 2019Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learningcited by 60
  12. 2020Reinforcement Learning in Economics and Financecited by 57
  13. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  14. 2019Risk-Averse Trust Region Optimization for Reward-Volatility Reductioncited by 53
  15. 2020Neuroevolution of Self-Interpretable Agentscited by 48
  16. 2017Interpretable Policies for Reinforcement Learning by Genetic Programmingcited by 47
  17. 2018Episodic Memory Deep Q-Networkscited by 41
  18. 2020Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitionscited by 41
  19. 2018Q-learning with Nearest Neighborscited by 37
  20. 2019Generation of Policy-Level Explanations for Reinforcement Learningcited by 37
  21. 2019Dot-to-Dot: Explainable Hierarchical Reinforcement Learning for Robotic Manipulationcited by 33
  22. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  23. 2022A Survey of Explainable Reinforcement Learningcited by 31
  24. 2021Explaining Latent Representations with a Corpus of Examplescited by 30
  25. 2019Policy Continuation with Hindsight Inverse Dynamicscited by 22
  26. 2021Generalizable Episodic Memory for Deep Reinforcement Learningcited by 22
  27. 2019Explaining Reinforcement Learning to Mere Mortals: An Empirical Studycited by 21
  28. 2019How Case Based Reasoning Explained Neural Networks: An XAI Survey of Post-Hoc Explanation-by-Example in ANN-CBR Twinscited by 20
  29. 2016Personalized Risk Scoring for Critical Care Prognosis using Mixtures of Gaussian Processescited by 19
  30. 2020TripleTree: A Versatile Interpretable Representation of Black Box Agents and their Environmentscited by 18
  31. 2019Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agentscited by 15
  32. 2021Safe Exploration by Solving Early Terminated MDPcited by 13
  33. 2021Inverse Contextual Bandits: Learning How Behavior Evolves over Timecited by 12
  34. 2021The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulationcited by 11
  35. 2023Neural Episodic Control with State Abstractioncited by 7
  36. 2022Value Function Decomposition for Iterative Design of Reinforcement Learning Agentscited by 6
  37. 2022Inferring Lexicographically-Ordered Rewards from Preferencescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 4; 2026: 1.

9 papers.

  1. 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruptioncited by 24
  2. 2024Tackling Data Corruption in Offline Reinforcement Learning via Sequence Modelingcited by 7
  3. 2025Offline RL with Smooth OOD Generalization in Convex Hull and its Neighborhoodcited by 3
  4. 2026Responsible Agentic AI Requires Explicit Provenancecited by 3
  5. 2024Defining Expertise: Applications to Treatment Effect Estimationcited by 2
  6. 2025Behaviour Discovery and Attribution for Explainable Reinforcement Learningcited by 2
  7. 2024To Switch or Not to Switch? Balanced Policy Switching in Offline Reinforcement Learningcited by 1
  8. 2025Explainable reinforcement learning from human feedback to improve alignmentcited by 1
  9. 2025Strategically Linked Decisions in Long-Term Planning and Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Tutorial: An Intuitive Explanation of Offline Reinforcement Learning Theory
  2. 2020Offline Reinforcement Learning Hands-On
  3. 2024An Investigation of Offline Reinforcement Learning in Factorisable Action Spaces
  4. 2021Representation Matters: Offline Pretraining for Sequential Decision Making
  5. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
  6. 2022Offline Policy Comparison with Confidence: Benchmarks and Baselines
  7. 2026Active Offline-to-Online Reinforcement Learning
  8. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism