# How much informative is your XAI? A decision-making assessment task to objectively measure the goodness of explanations

arXiv: https://arxiv.org/abs/2312.04379 · HTML: https://mlsanity.com/paper/2312.04379/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 30 references that are in the map.
- First posted on arXiv 7 Dec 2023.

## Builds on (most cited first, 10 of 10)

- [Counterfactual Visual Explanations](https://mlsanity.com/paper/1904.07451/) (2019)
- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Metrics for Explainable AI: Challenges and Prospects](https://mlsanity.com/paper/1812.04608/) (2018)
- [Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations](https://mlsanity.com/paper/1912.09024/) (2019)
- [SCOUT: Self-aware Discriminant Counterfactual Explanations](https://mlsanity.com/paper/2004.07769/) (2020)
- [Conservative Q-Improvement: Reinforcement Learning for an Interpretable Decision-Tree Policy](https://mlsanity.com/paper/1907.01180/) (2019)
- [Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2204.06916/) (2022)
- [A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems](https://mlsanity.com/paper/1811.11839/) (2018)
- [The human-AI relationship in decision-making: AI explanation to support people on justifying their decisions](https://mlsanity.com/paper/2102.05460/) (2021)
- [A User-Centred Framework for Explainable Artificial Intelligence in Human-Robot Interaction](https://mlsanity.com/paper/2109.12912/) (2021)

## Built on by (most cited first, 1 of 1)

- [Study on the Helpfulness of Explainable Artificial Intelligence](https://mlsanity.com/paper/2410.11896/) (2024)

## Abstract (arXiv)

There is an increasing consensus about the effectiveness of user-centred approaches in the explainable artificial intelligence (XAI) field. Indeed, the number and complexity of personalised and user-centred approaches to XAI have rapidly grown in recent years. Often, these works have a two-fold objective: (1) proposing novel XAI techniques able to consider the users and (2) assessing the textitgoodness of such techniques with respect to others. From these new works, it emerged that user-centred approaches to XAI positively affect the interaction between users and systems. However, so far, the goodness of XAI systems has been measured through indirect measures, such as performance. In this paper, we propose an assessment task to objectively and quantitatively measure the goodness of XAI systems in terms of their textitinformation power, which we intended as the amount of information the system provides to the users during the interaction. Moreover, we plan to use our task to objectively compare two XAI techniques in a human-robot decision-making task to understand deeper whether user-centred approaches are more informative than classical ones.