# Understanding the Challenges and Opportunities of Generative AI Apps: An Empirical Study

arXiv: https://arxiv.org/abs/2506.16453 · HTML: https://mlsanity.com/paper/2506.16453/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2025.

## Built on by (most cited first, 3 of 3)

- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [Collaborator or Assistant? How AI Coding Agents Partition Work Across Pull Request Lifecycles](https://mlsanity.com/paper/2605.08017/) (2026)
- [From Assistance to Agency: Rethinking Autonomy and Control in CI/CD Pipelines](https://mlsanity.com/paper/2605.07062/) (2026)

## Abstract (arXiv)

Generative AI (Gen-AI) is increasingly integrated into mobile applications (apps), introducing new capabilities while also creating new challenges for users. However, despite their growing adoption, we lack an ecosystem-level understanding of the experiences, opportunities, and challenges users report across Gen-AI mobile apps. We conduct a user-centered analysis of 1,035,342 reviews from 171 Gen-AI apps from the Google Play Store. We propose SARA (Selection, Acquisition, Refinement, and Analysis), a four-phase framework that leverages prompt-based LLMs for large-scale review analysis. We validate the reliability of LLM-based topic extraction and assignment using 4,353 manually evaluated reviews, achieving 91% accuracy with five-shot prompting and filtering of non-informative reviews. We identify the top ten topics (e.g., AI Performance and Emotional Connection) and perform a cross-platform comparison with Apple App Store reviews. Through qualitative analysis of 762 reviews, we uncover three opportunities (AI for Accessibility and Wellbeing, AI as a Collaborative Creative Tool, and AI Versatility) and three challenges (Managing User Expectations and AI Limitations, Balancing Content Moderation and Creative Freedom, and Strategic Integration of Gen-AI Features). Finally, we analyze temporal trends, revealing how the topics discussed and their evaluations change over time, including changing concerns around emotional connection and content moderation.