# Whom do Explanations Serve? A Systematic Literature Survey of User Characteristics in Explainable Recommender Systems Evaluation

arXiv: https://arxiv.org/abs/2412.14193 · HTML: https://mlsanity.com/paper/2412.14193/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 158 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 19 of 19)

- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [Explainable Recommendation: A Survey and New Perspectives](https://mlsanity.com/paper/1804.11192/) (2018)
- [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020)
- [What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research](https://mlsanity.com/paper/2102.07817/) (2021)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- [Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning](https://mlsanity.com/paper/2202.05402/) (2022)
- [A systematic review and taxonomy of explanations in decision support and recommender systems](https://mlsanity.com/paper/2006.08672/) (2020)
- [Explainable Recommendation via Multi-Task Learning in Opinionated Text Data](https://mlsanity.com/paper/1806.03568/) (2018)
- [Why? Why not? When? Visual Explanations of Agent Behavior in Reinforcement Learning](https://mlsanity.com/paper/2104.02818/) (2021)
- [Is More Always Better? The Effects of Personal Characteristics and Level of Detail on the Perception of Explanations in a Recommender System](https://mlsanity.com/paper/2304.00969/) (2023)
- [Providing Explanations for Recommendations in Reciprocal Environments](https://mlsanity.com/paper/1807.01227/) (2018)
- [Effects of interactivity and presentation on review-based explanations for recommendations](https://mlsanity.com/paper/2105.11794/) (2021)
- [Exploring The Role of Local and Global Explanations in Recommender Systems](https://mlsanity.com/paper/2109.13301/) (2021)
- [Decision Making of Maximizers and Satisficers Based on Collaborative Explanations](https://mlsanity.com/paper/1805.11537/) (2018)
- [Improved Explanatory Efficacy on Human Affect and Workload through Interactive Process in Artificial Intelligence](https://mlsanity.com/paper/1912.07416/) (2019)
- [Exploring Users' Perception of Collaborative Explanation Styles](https://mlsanity.com/paper/1805.00977/) (2018)

## Built on by (most cited first, 1 of 1)

- [The Utility of LLMs in Recommender Systems Explanation Evaluation](https://mlsanity.com/paper/2609.01627/) (2026)

## Abstract (arXiv)

Adding explanations to recommender systems is said to have multiple benefits, such as increasing user trust or system transparency. Previous work from other application areas suggests that specific user characteristics impact the users' perception of the explanation. However, we rarely find this type of evaluation for recommender systems explanations. This paper addresses this gap by surveying 124 papers in which recommender systems explanations were evaluated in user studies. We analyzed their participant descriptions and study results where the impact of user characteristics on the explanation effects was measured. Our findings suggest that the results from the surveyed studies predominantly cover specific users who do not necessarily represent the users of recommender systems in the evaluation domain. This may seriously hamper the generalizability of any insights we may gain from current studies on explanations in recommender systems. We further find inconsistencies in the data reporting, which impacts the reproducibility of the reported results. Hence, we recommend actions to move toward a more inclusive and reproducible evaluation.