mlsanityData to 14 Oct 2026

Evaluating User Experience in Conversational Recommender Systems: A Systematic Review Across Classical and LLM-Powered Approaches

arXiv 2508.02096 ↗First posted 4 Aug 2025cs.IR cs.AI cs.HCcited by 1

Not on the map. Why?

Abstract (arXiv)

Conversational Recommender Systems (CRSs) are receiving growing research attention across domains, yet their user experience (UX) evaluation remains limited. Existing reviews largely overlook empirical UX studies, particularly in adaptive and large language model (LLM)-based CRSs. To address this gap, we conducted a systematic review following PRISMA guidelines, synthesising 23 empirical studies published between 2017 and 2025. We analysed how UX has been conceptualised, measured, and shaped by domain, adaptivity, and LLM. Our findings reveal persistent limitations: post hoc surveys dominate, turn-level affective UX constructs are rarely assessed, and adaptive behaviours are seldom linked to UX outcomes. LLM-based CRSs introduce further challenges, including epistemic opacity and verbosity, yet evaluations infrequently address these issues. We contribute a structured synthesis of UX metrics, a comparative analysis of adaptive and nonadaptive systems, and a forward-looking agenda for LLM-aware UX evaluation. These findings support the development of more transparent, engaging, and user-centred CRS evaluation practices.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 14 of its 57 references that are in the map. First posted on arXiv 4 Aug 2025.

Builds on

14 of 57 references.

References · 14

  1. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  3. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  4. 2023A Survey on Large Language Models for Recommendationcited by 379
  5. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 309
  6. 2018The Design and Implementation of XiaoIce, an Empathetic Social Chatbotcited by 172
  7. 2020A Survey on Conversational Recommender Systemscited by 143
  8. 2021Advances and Challenges in Conversational Recommender Systems: A Surveycited by 118
  9. 2024A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)cited by 58
  10. 2024RecAI: Leveraging Large Language Models for Next-Generation Recommender Systemscited by 20
  11. 2022Impacts of Personal Characteristics on User Trust in Conversational Recommender Systemscited by 8
  12. 2022Evaluating Conversational Recommender Systems: A Landscape of Researchcited by 7
2 more
  1. 2025User Experience with LLM-powered Conversational Recommendation Systems: A Case of Music Recommendationcited by 6
  2. 2024Navigating User Experience of ChatGPT-based Conversational Recommender Systems: The Effects of Prompt Guidance and Recommendation Domaincited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026How Personal Characteristics Shape User Exploration of Diverse Movie Recommendations with a LLM-Based Multi-Agent Systemcited by 0