# Reverse Engineering User Stories from Code using Large Language Models

arXiv: https://arxiv.org/abs/2509.19587 · HTML: https://mlsanity.com/paper/2509.19587/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 44 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Structured Chain-of-Thought Prompting for Code Generation](https://mlsanity.com/paper/2305.06599/) (2023)
- [Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review](https://mlsanity.com/paper/2307.02503/) (2023)
- [Source Code Summarization in the Era of Large Language Models](https://mlsanity.com/paper/2407.07959/) (2024)
- [Project-Level Encoding for Neural Source Code Summarization of Subroutines](https://mlsanity.com/paper/2103.11599/) (2021)
- [Ensemble Models for Neural Source Code Summarization of Subroutines](https://mlsanity.com/paper/2107.11423/) (2021)
- [Automated User Story Generation with Test Case Specification Using Large Language Model](https://mlsanity.com/paper/2404.01558/) (2024)
- [The GitHub Recent Bugs Dataset for Evaluating LLM-based Debugging Applications](https://mlsanity.com/paper/2310.13229/) (2023)
- [Decoding Logic Errors: A Comparative Study on Bug Detection by Students and Large Language Models](https://mlsanity.com/paper/2311.16017/) (2023)
- [Perceptions of the State of D&I and D&I Initiative in the ASF](https://mlsanity.com/paper/2202.13260/) (2022)

## Built on by (most cited first, 2 of 2)

- [Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures](https://mlsanity.com/paper/2601.03988/) (2026)
- [Single-Channel EEG-Based Cognitive Load Assessment in Online Learning: A Hybrid Deep Learning Approach](https://mlsanity.com/paper/2607.01795/) (2026)

## Abstract (arXiv)

User stories are essential in agile development, yet often missing or outdated in legacy and poorly documented systems. We investigate whether large language models (LLMs) can automatically recover user stories directly from source code and how prompt design impacts output quality. Using 1,750 annotated C++ snippets of varying complexity, we evaluate five state-of-the-art LLMs across six prompting strategies. Results show that all models achieve, on average, an F1 score of 0.8 for code up to 200 NLOC. Our findings show that a single illustrative example enables the smallest model (8B) to match the performance of a much larger 70B model. In contrast, structured reasoning via Chain-of-Thought offers only marginal gains, primarily for larger models.