# Leveraging LLMs for User Stories in AI Systems: UStAI Dataset

arXiv: https://arxiv.org/abs/2504.00513 · HTML: https://mlsanity.com/paper/2504.00513/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 36 references that are in the map.
- First posted on arXiv 1 Apr 2025.

## Builds on (most cited first, 8 of 8)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [Software Engineering for AI-Based Systems: A Survey](https://mlsanity.com/paper/2105.01984/) (2021)
- [Algorithmic Fairness Datasets: the Story so Far](https://mlsanity.com/paper/2202.01711/) (2022)
- [MedSyn: LLM-based Synthetic Medical Text Generation Framework](https://mlsanity.com/paper/2408.02056/) (2024)
- [AI for All: Operationalising Diversity and Inclusion Requirements for AI Systems](https://mlsanity.com/paper/2311.14695/) (2023)
- [The REquirements TRacing On target (RETRO).NET Dataset](https://mlsanity.com/paper/1807.11344/) (2018)

## Built on by (most cited first, 4 of 4)

- [Multi-Agent LLMs as Ethics Advocates for AI-Based Systems](https://mlsanity.com/paper/2507.08392/) (2025)
- [Automated Alignment between Elicitation Interviews and Requirements](https://mlsanity.com/paper/2510.08622/) (2025)
- [Are We Aligned? A Preliminary Investigation of the Alignment of Responsible AI Values between LLMs and Human Judgment](https://mlsanity.com/paper/2511.04157/) (2025)
- [From Reviews to Requirements: Can LLMs Generate Human-Like User Stories?](https://mlsanity.com/paper/2603.28163/) (2026)

## Abstract (arXiv)

AI systems are gaining widespread adoption across various sectors and domains. Creating high-quality AI system requirements is crucial for aligning the AI system with business goals and consumer values and for social responsibility. However, with the uncertain nature of AI systems and the heavy reliance on sensitive data, more research is needed to address the elicitation and analysis of AI systems requirements. With the proprietary nature of many AI systems, there is a lack of open-source requirements artifacts and technical requirements documents for AI systems, limiting broader research and investigation. With Large Language Models (LLMs) emerging as a promising alternative to human-generated text, this paper investigates the potential use of LLMs to generate user stories for AI systems based on abstracts from scholarly papers. We conducted an empirical evaluation using three LLMs and generated 1260 user stories from 42 abstracts from 26 domains. We assess their quality using the Quality User Story (QUS) framework. Moreover, we identify relevant non-functional requirements (NFRs) and ethical principles. Our analysis demonstrates that the investigated LLMs can generate user stories inspired by the needs of various stakeholders, offering a promising approach for generating user stories for research purposes and for aiding in the early requirements elicitation phase of AI systems. We have compiled and curated a collection of stories generated by various LLMs into a dataset (UStAI), which is now publicly available for use.