Human-AI Interaction and User Satisfaction: Empirical Evidence from Online Reviews of AI Products
Not on the map. Why?
Abstract (arXiv)
Human-AI Interaction (HAI) guidelines and design principles have become increasingly important in both industry and academia to guide the development of AI systems that align with user needs and expectations. However, large-scale empirical evidence on how HAI principles shape user satisfaction in practice remains limited. This study addresses that gap by analyzing over 100,000 user reviews of AI-related products from G2, a leading review platform for business software and services. Based on widely adopted industry guidelines, we identify seven core HAI dimensions and examine their coverage and sentiment within the reviews. We find that the sentiment on four HAI dimensions-adaptability, customization, error recovery, and security-is positively associated with overall user satisfaction. Moreover, we show that engagement with HAI dimensions varies by professional background: Users with technical job roles are more likely to discuss system-focused aspects, such as reliability, while non-technical users emphasize interaction-focused features like customization and feedback. Interestingly, the relationship between HAI sentiment and overall satisfaction is not moderated by job role, suggesting that once an HAI dimension has been identified by users, its effect on satisfaction is consistent across job roles.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 14 of its 67 references that are in the map. First posted on arXiv 23 Mar 2025.
Builds on
14 of 67 references.
References · 14
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
- 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
- 2020Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthycited by 183
- 2024From Explainable to Interactive AI: A Literature Review on Current Trends in Human-AI Interactioncited by 26
- 2022Trusted AI in Multi-agent Systems: An Overview of Privacy and Security for Distributed Learningcited by 22
- 2023Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebookcited by 22
- 2020A Comparative Analysis of Industry Human-AI Interaction Guidelinescited by 10
- 2021Towards a Robust and Trustworthy Machine Learning System Development: An Engineering Perspectivecited by 9
- 2023A Holistic Assessment of the Reliability of Machine Learning Systemscited by 9
- 2023Requirements Practices and Gaps When Engineering Human-Centered Artificial Intelligence Systemscited by 8
- 2025LLM Content Moderation and User Satisfaction: Evidence from Response Refusals in Chatbot Arenacited by 6
Built on by
Citing papers per year: 2025: 1.
1 paper.