# From Reviews to Requirements: Can LLMs Generate Human-Like User Stories?

arXiv: https://arxiv.org/abs/2603.28163 · HTML: https://mlsanity.com/paper/2603.28163/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 24 references that are in the map.
- First posted on arXiv 30 Mar 2026.

## Builds on (most cited first, 6 of 6)

- [Advancing Requirements Engineering through Generative AI: Assessing the Role of LLMs](https://mlsanity.com/paper/2310.13976/) (2023)
- [Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviews](https://mlsanity.com/paper/2104.05861/) (2021)
- [Automated User Story Generation with Test Case Specification Using Large Language Model](https://mlsanity.com/paper/2404.01558/) (2024)
- [Leveraging LLMs for User Stories in AI Systems: UStAI Dataset](https://mlsanity.com/paper/2504.00513/) (2025)
- [Zero-shot Bilingual App Reviews Mining with Large Language Models](https://mlsanity.com/paper/2311.03058/) (2023)
- [Exploring LLMs Impact on Student-Created User Stories and Acceptance Testing in Software Development](https://mlsanity.com/paper/2502.02675/) (2025)

## Abstract (arXiv)

App store reviews provide a constant flow of real user feedback that can help improve software requirements. However, these reviews are often messy, informal, and difficult to analyze manually at scale. Although automated techniques exist, many do not perform well when replicated and often fail to produce clean, backlog-ready user stories for agile projects. In this study, we evaluate how well large language models (LLMs) such as GPT-3.5 Turbo, Gemini 2.0 Flash, and Mistral 7B Instruct can generate usable user stories directly from raw app reviews. Using the Mini-BAR dataset of 1,000+ health app reviews, we tested zero-shot, one-shot, and two-shot prompting methods. We evaluated the generated user stories using both human judgment (via the RUST framework) and a RoBERTa classifier fine-tuned on UStAI to assess their overall quality. Our results show that LLMs can match or even outperform humans in writing fluent, well-formatted user stories, especially when few-shot prompts are used. However, they still struggle to produce independent and unique user stories, which are essential for building a strong agile backlog. Overall, our findings show how LLMs can reliably turn unstructured app reviews into actionable software requirements, providing developers with clear guidance to turn user feedback into meaningful improvements.