A Case Study Investigating the Role of Generative AI in Quality Evaluations of Epics in Agile Software Development
Not on the map. Why?
Abstract (arXiv)
The broad availability of generative AI offers new opportunities to support various work domains, including agile software development. Agile epics are a key artifact for product managers to communicate requirements to stakeholders. However, in practice, they are often poorly defined, leading to churn, delivery delays, and cost overruns. In this industry case study, we investigate opportunities for large language models (LLMs) to evaluate agile epic quality in a global company. Results from a user study with 17 product managers indicate how LLM evaluations could be integrated into their work practices, including perceived values and usage in improving their epics. High levels of satisfaction indicate that agile epics are a new, viable application of AI evaluations. However, our findings also outline challenges, limitations, and adoption barriers that can inform both practitioners and researchers on the integration of such evaluations into future agile work practices.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 19 of its 78 references that are in the map. First posted on arXiv 12 May 2025.
Builds on
19 of 78 references.
References · 19
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
- 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
- 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 575
- 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
- 2023EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteriacited by 74
- 2024AI-Augmented Brainwriting: Investigating the use of LLMs in group ideationcited by 44
- 2024A Taxonomy for Human-LLM Interaction Modes: An Initial Explorationcited by 38
- 2024Human-Centered Design Recommendations for LLM-as-a-Judgecited by 37
- 2023Investigating ChatGPT's Potential to Assist in Requirements Elicitation Processescited by 25
- 2024Large Language Model as an Assignment Evaluator: Insights, Feedback, and Challenges in a 1000+ Student Coursecited by 19
- 2024Black-box Uncertainty Quantification Method for LLM-as-a-Judgecited by 18
7 more
- 2024Leveraging LLMs for the Quality Assurance of Software Requirementscited by 15
- 2023ChatCoder: Chat-based Refine Requirement Improves LLMs' Code Generationcited by 13
- 2024AI based Multiagent Approach for Requirements Elicitation and Analysiscited by 6
- 2024LLMs as Evaluators: A Novel Approach to Evaluate Bug Report Summarizationcited by 6
- 2025Generative AI Uses and Risks for Knowledge Workers in a Science Organizationcited by 5
- 2024A Survey on Large Language Models from Concept to Implementationcited by 4
- 2020Documentation of quality requirements in agile software developmentcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.