# What Users Value and Critique: Large-Scale Analysis of User Feedback on AI-Powered Mobile Apps

arXiv: https://arxiv.org/abs/2506.10785 · HTML: https://mlsanity.com/paper/2506.10785/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 50 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Knowing What, How and Why: A Near Complete Solution for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1911.01616/) (2019)
- [A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges](https://mlsanity.com/paper/2203.01054/) (2022)
- [App-Aware Response Synthesis for User Reviews](https://mlsanity.com/paper/2007.15793/) (2020)
- [The Product Beyond the Model -- An Empirical Study of Repositories of Open-Source ML Products](https://mlsanity.com/paper/2308.04328/) (2023)
- [Mining User Opinions in Mobile App Reviews: A Keyword-based Approach](https://mlsanity.com/paper/1505.04657/) (2015)
- [Automating App Review Response Generation](https://mlsanity.com/paper/2002.03552/) (2020)
- [Automating App Review Response Generation Based on Contextual Knowledge](https://mlsanity.com/paper/2010.06301/) (2020)
- [Listening to Users' Voice: Automatic Summarization of Helpful App Reviews](https://mlsanity.com/paper/2210.06235/) (2022)
- [Finding the Needle in a Haystack: On the Automatic Identification of Accessibility User Reviews](https://mlsanity.com/paper/2210.09947/) (2022)
- [Fairness Concerns in App Reviews: A Study on AI-based Mobile Apps](https://mlsanity.com/paper/2401.08097/) (2024)
- [Proactive Prioritization of App Issues via Contrastive Learning](https://mlsanity.com/paper/2303.06586/) (2023)
- [A Decade of Privacy-Relevant Android App Reviews: Large Scale Trends](https://mlsanity.com/paper/2403.02292/) (2024)
- [An Empirical Study of AI Techniques in Mobile Applications](https://mlsanity.com/paper/2212.01635/) (2022)

## Abstract (arXiv)

Artificial Intelligence (AI)-powered features have rapidly proliferated across mobile apps in various domains, including productivity, education, entertainment, and creativity. However, how users perceive, evaluate, and critique these AI features remains largely unexplored, primarily due to the overwhelming volume of user feedback. In this work, we present the first comprehensive, large-scale study of user feedback on AI-powered mobile apps, leveraging a curated dataset of 292 AI-driven apps across 14 categories with 894K AI-specific reviews from Google Play. We develop and validate a multi-stage analysis pipeline that begins with a human-labeled benchmark and systematically evaluates large language models (LLMs) and prompting strategies. Each stage, including review classification, aspect-sentiment extraction, and clustering, is validated for accuracy and consistency. Our pipeline enables scalable, high-precision analysis of user feedback, extracting over one million aspect-sentiment pairs clustered into 18 positive and 15 negative user topics. Our analysis reveals that users consistently focus on a narrow set of themes: positive comments emphasize productivity, reliability, and personalized assistance, while negative feedback highlights technical failures (e.g., scanning and recognition), pricing concerns, and limitations in language support. Our pipeline surfaces both satisfaction with one feature and frustration with another within the same review. These fine-grained, co-occurring sentiments are often missed by traditional approaches that treat positive and negative feedback in isolation or rely on coarse-grained analysis. To this end, our approach provides a more faithful reflection of the real-world user experiences with AI-powered apps. Category-aware analysis further uncovers both universal drivers of satisfaction and domain-specific frustrations.