# HEX: Human-in-the-loop Explainability via Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2206.01343 · HTML: https://mlsanity.com/paper/2206.01343/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 87 references that are in the map.
- First posted on arXiv 2 Jun 2022.

## Builds on (most cited first, 30 of 30)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Methods for Interpreting and Understanding Deep Neural Networks](https://mlsanity.com/paper/1706.07979/) (2017)
- [Distilling a Neural Network Into a Soft Decision Tree](https://mlsanity.com/paper/1711.09784/) (2017)
- [A Survey on the Explainability of Supervised Machine Learning](https://mlsanity.com/paper/2011.07876/) (2020)
- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [Learning how to explain neural networks: PatternNet and PatternAttribution](https://mlsanity.com/paper/1705.05598/) (2017)
- [Model Agnostic Supervised Local Explanations](https://mlsanity.com/paper/1807.02910/) (2018)
- [The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification](https://mlsanity.com/paper/1503.01161/) (2015)
- [Classification Under Human Assistance](https://mlsanity.com/paper/2006.11845/) (2020)
- [Auditing Black-box Models for Indirect Influence](https://mlsanity.com/paper/1602.07043/) (2016)
- [Shapley Explanation Networks](https://mlsanity.com/paper/2104.02297/) (2021)
- [Counterfactual Explanations in Sequential Decision Making Under Uncertainty](https://mlsanity.com/paper/2107.02776/) (2021)
- [Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach](https://mlsanity.com/paper/1606.09066/) (2016)
- [One-Shot Federated Learning: Theoretical Limits and Algorithms to Achieve Them](https://mlsanity.com/paper/1905.04634/) (2019)
- [Human-Centered Concept Explanations for Neural Networks](https://mlsanity.com/paper/2202.12451/) (2022)
- [Generalized Inverse Classification](https://mlsanity.com/paper/1610.01675/) (2016)
- [Model-agnostic and Scalable Counterfactual Explanations via Reinforcement Learning](https://mlsanity.com/paper/2106.02597/) (2021)
- [Flexible and Context-Specific AI Explainability: A Multidisciplinary Approach](https://mlsanity.com/paper/2003.07703/) (2020)
- [Brittle AI, Causal Confusion, and Bad Mental Models: Challenges and Successes in the XAI Program](https://mlsanity.com/paper/2106.05506/) (2021)
- [A Unified Study of Machine Learning Explanation Evaluation Metrics](https://mlsanity.com/paper/2203.14265/) (2022)
- [A budget-constrained inverse classification framework for smooth classifiers](https://mlsanity.com/paper/1605.09068/) (2016)
- [Early Predictions of Movie Success: the Who, What, and When of Profitability](https://mlsanity.com/paper/1506.05382/) (2015)

## Abstract (arXiv)

The use of machine learning (ML) models in decision-making contexts, particularly those used in high-stakes decision-making, are fraught with issue and peril since a person - not a machine - must ultimately be held accountable for the consequences of the decisions made using such systems. Machine learning explainability (MLX) promises to provide decision-makers with prediction-specific rationale, assuring them that the model-elicited predictions are made for the right reasons and are thus reliable. Few works explicitly consider this key human-in-the-loop (HITL) component, however. In this work we propose HEX, a human-in-the-loop deep reinforcement learning approach to MLX. HEX incorporates 0-distrust projection to synthesize decider specific explanation-providing policies from any arbitrary classification model. HEX is also constructed to operate in limited or reduced training data scenarios, such as those employing federated learning. Our formulation explicitly considers the decision boundary of the ML model in question, rather than the underlying training data, which is a shortcoming of many model-agnostic MLX methods. Our proposed methods thus synthesize HITL MLX policies that explicitly capture the decision boundary of the model in question for use in limited data scenarios.