# Using LLMs in Software Requirements Specifications: An Empirical Evaluation

arXiv: https://arxiv.org/abs/2404.17842 · HTML: https://mlsanity.com/paper/2404.17842/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 39 references that are in the map.
- First posted on arXiv 27 Apr 2024.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Large Language Model Guided Tree-of-Thought](https://mlsanity.com/paper/2305.08291/) (2023)
- [Investigating Explainability of Generative AI for Code through Scenario-based Design](https://mlsanity.com/paper/2202.04903/) (2022)
- [Advancing Requirements Engineering through Generative AI: Assessing the Role of LLMs](https://mlsanity.com/paper/2310.13976/) (2023)
- [A Survey on Deep Learning for Software Engineering](https://mlsanity.com/paper/2011.14597/) (2020)
- [Large Language Models Based Automatic Synthesis of Software Specifications](https://mlsanity.com/paper/2304.09181/) (2023)
- [Automated User Story Generation with Test Case Specification Using Large Language Model](https://mlsanity.com/paper/2404.01558/) (2024)

## Built on by (most cited first, 20 of 20)

- [Large Language Model-Based Agents for Software Engineering: A Survey](https://mlsanity.com/paper/2409.02977/) (2024)
- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models](https://mlsanity.com/paper/2502.14302/) (2025)
- [Leveraging Graph-RAG and Prompt Engineering to Enhance LLM-Based Automated Requirement Traceability and Compliance Checks](https://mlsanity.com/paper/2412.08593/) (2024)
- [Teaching with Lies: Curriculum DPO on Synthetic Negatives for Hallucination Detection](https://mlsanity.com/paper/2505.17558/) (2025)
- [Can LLMs Generate User Stories and Assess Their Quality?](https://mlsanity.com/paper/2507.15157/) (2025)
- [ReqInOne: A Large Language Model-Based Agent for Software Requirements Specification Generation](https://mlsanity.com/paper/2508.09648/) (2025)
- [A Blueprint for AI-Driven Software Quality: Integrating LLMs with Established Standards](https://mlsanity.com/paper/2505.13766/) (2025)
- [Reversa: A Reverse Documentation Engineering Framework for Converting Legacy Software into Operational Specifications for AI Agents](https://mlsanity.com/paper/2605.18684/) (2026)
- [From Inductive to Deductive: LLMs-Based Qualitative Data Analysis in Requirements Engineering](https://mlsanity.com/paper/2504.19384/) (2025)
- [AI in Money Matters](https://mlsanity.com/paper/2505.07393/) (2025)
- [LLM assisted web application functional requirements generation: A case study of four popular LLMs over a Mess Management System](https://mlsanity.com/paper/2505.18019/) (2025)
- [Knowledge-Guided Multi-Agent Framework for Automated Requirements Development: A Vision](https://mlsanity.com/paper/2506.22656/) (2025)
- [Generating Privacy Stories From Software Documentation](https://mlsanity.com/paper/2506.23014/) (2025)
- [Adoption of Generative Artificial Intelligence in the German Software Engineering Industry: An Empirical Study](https://mlsanity.com/paper/2601.16700/) (2026)
- [DeepQuali: Initial results of a study on the use of large language models for assessing the quality of user stories](https://mlsanity.com/paper/2602.08887/) (2026)
- [A systematic review of generative AI usage for IT project management](https://mlsanity.com/paper/2604.21958/) (2026)
- [PrivacyAkinator: Articulating Key Privacy Design Decisions by Answering LLM-Generated Multiple-choice Questions](https://mlsanity.com/paper/2605.20206/) (2026)
- [ReqGenX: An Empirical Study of Atomic Decomposition, Artifact Regeneration, and Reconstruction for Legacy SRS Documents](https://mlsanity.com/paper/2607.16564/) (2026)
- [Relationally Guided Use Case Modeling with LLMs](https://mlsanity.com/paper/2609.18291/) (2026)

## Abstract (arXiv)

The creation of a Software Requirements Specification (SRS) document is important for any software development project. Given the recent prowess of Large Language Models (LLMs) in answering natural language queries and generating sophisticated textual outputs, our study explores their capability to produce accurate, coherent, and structured drafts of these documents to accelerate the software development lifecycle. We assess the performance of GPT-4 and CodeLlama in drafting an SRS for a university club management system and compare it against human benchmarks using eight distinct criteria. Our results suggest that LLMs can match the output quality of an entry-level software engineer to generate an SRS, delivering complete and consistent drafts. We also evaluate the capabilities of LLMs to identify and rectify problems in a given requirements document. Our experiments indicate that GPT-4 is capable of identifying issues and giving constructive feedback for rectifying them, while CodeLlama's results for validation were not as encouraging. We repeated the generation exercise for four distinct use cases to study the time saved by employing LLMs for SRS generation. The experiment demonstrates that LLMs may facilitate a significant reduction in development time for entry-level software engineers. Hence, we conclude that the LLMs can be gainfully used by software engineers to increase productivity by saving time and effort in generating, validating and rectifying software requirements.