# LLM-based agents for automating the enhancement of user story quality: An early report

arXiv: https://arxiv.org/abs/2403.09442 · HTML: https://mlsanity.com/paper/2403.09442/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 10 of 10)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediations](https://mlsanity.com/paper/2310.04988/) (2023)
- [ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design](https://mlsanity.com/paper/2303.07839/) (2023)
- [Investigating ChatGPT's Potential to Assist in Requirements Elicitation Processes](https://mlsanity.com/paper/2307.07381/) (2023)
- [Generative Artificial Intelligence for Software Engineering -- A Research Agenda](https://mlsanity.com/paper/2310.18648/) (2023)

## Built on by (most cited first, 13 of 13)

- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [Can LLMs "Reason" in Music? An Evaluation of LLMs' Capability of Music Understanding and Generation](https://mlsanity.com/paper/2407.21531/) (2024)
- [Facilitating Trustworthy Human-Agent Collaboration in LLM-based Multi-Agent System oriented Software Engineering](https://mlsanity.com/paper/2505.04251/) (2025)
- [Experimenting with Multi-Agent Software Development: Towards a Unified Platform](https://mlsanity.com/paper/2406.05381/) (2024)
- [From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development -- An Opinion Paper](https://mlsanity.com/paper/2503.07450/) (2025)
- [Getting Inspiration for Feature Elicitation: App Store- vs. LLM-based Approach](https://mlsanity.com/paper/2408.17404/) (2024)
- [AI and Agile Software Development: From Frustration to Success -- XP2025 Workshop Summary](https://mlsanity.com/paper/2506.20159/) (2025)
- [Multi-Agent LLMs as Ethics Advocates for AI-Based Systems](https://mlsanity.com/paper/2507.08392/) (2025)
- [Generating Privacy Stories From Software Documentation](https://mlsanity.com/paper/2506.23014/) (2025)
- [AI and Agile Software Development: A Research Roadmap from the XP2025 Workshop](https://mlsanity.com/paper/2508.20563/) (2025)
- [Comparing Large Language Models on Scrum Certification-Style Questions: Accuracy, Stability, and Error Patterns](https://mlsanity.com/paper/2607.00048/) (2026)
- [Prompting GPT-5 on Scrum Certification Questions: An Empirical Accuracy Study](https://mlsanity.com/paper/2607.00049/) (2026)
- [Integrating AI into Requirements Quality Learning in Software Engineering Education: A TPACK-Guided Empirical Study](https://mlsanity.com/paper/2607.28176/) (2026)

## Abstract (arXiv)

In agile software development, maintaining high-quality user stories is crucial, but also challenging. This study explores the use of large language models to automatically improve the user story quality in Austrian Post Group IT agile teams. We developed a reference model for an Autonomous LLM-based Agent System and implemented it at the company. The quality of user stories in the study and the effectiveness of these agents for user story quality improvement was assessed by 11 participants across six agile teams. Our findings demonstrate the potential of LLMs in improving user story quality, contributing to the research on AI role in agile development, and providing a practical example of the transformative impact of AI in an industry setting.