mlsanityData to 14 Oct 2026

Reverse Engineering User Stories from Code using Large Language Models

arXiv 2509.19587 ↗First posted 23 Sep 2025cs.SE cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

User stories are essential in agile development, yet often missing or outdated in legacy and poorly documented systems. We investigate whether large language models (LLMs) can automatically recover user stories directly from source code and how prompt design impacts output quality. Using 1,750 annotated C++ snippets of varying complexity, we evaluate five state-of-the-art LLMs across six prompting strategies. Results show that all models achieve, on average, an F1 score of 0.8 for code up to 200 NLOC. Our findings show that a single illustrative example enables the smallest model (8B) to match the performance of a much larger 70B model. In contrast, structured reasoning via Chain-of-Thought offers only marginal gains, primarily for larger models.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 10 of its 44 references that are in the map. First posted on arXiv 23 Sep 2025.

Builds on

10 of 44 references.

References · 10

  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  2. 2023Structured Chain-of-Thought Prompting for Code Generationcited by 138
  3. 2023Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Reviewcited by 32
  4. 2024Source Code Summarization in the Era of Large Language Modelscited by 31
  5. 2021Project-Level Encoding for Neural Source Code Summarization of Subroutinescited by 24
  6. 2021Ensemble Models for Neural Source Code Summarization of Subroutinescited by 12
  7. 2024Automated User Story Generation with Test Case Specification Using Large Language Modelcited by 7
  8. 2023The GitHub Recent Bugs Dataset for Evaluating LLM-based Debugging Applicationscited by 6
  9. 2023Decoding Logic Errors: A Comparative Study on Bug Detection by Students and Large Language Modelscited by 6
  10. 2022Perceptions of the State of D&I and D&I Initiative in the ASFcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024The Impact of Prompt Programming on Function-Level Code Generation
  2. 2026Prompt-Driven Code Summarization: A Systematic Literature Review
  3. 2026AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation
  4. 2025Code Red! On the Harmfulness of Applying Off-the-shelf Large Language Models to Programming Tasks
  5. 2023SelfEvolve: A Code Evolution Framework via Large Language Models
  6. 2023On Extracting Specialized Code Abilities from Large Language Models: A Feasibility Study
  7. 2025Towards Effectively Leveraging Execution Traces for Program Repair with Code LLMs
  8. 2026Large Language Models for Programming: Actually Fixing or Reimplementing Incorrect Code?