mlsanityData to 7 Oct 2026

Towards Requirements Engineering for RAG Systems

arXiv 2505.07553 ↗First posted 12 May 2025cs.SE cs.AIcited by 1

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

This short paper explores how a maritime company develops and integrates large-language models (LLM). Specifically by looking at the requirements engineering for Retrieval Augmented Generation (RAG) systems in expert settings. Through a case study at a maritime service provider, we demonstrate how data scientists face a fundamental tension between user expectations of AI perfection and the correctness of the generated outputs. Our findings reveal that data scientists must identify context-specific "retrieval requirements" through iterative experimentation together with users because they are the ones who can determine correctness. We present an empirical process model describing how data scientists practically elicited these "retrieval requirements" and managed system limitations. This work advances software engineering knowledge by providing insights into the specialized requirements engineering processes for implementing RAG systems in complex domain-specific applications.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 12 of its 32 references that are in the map. First posted on arXiv 12 May 2025.

Builds on

12 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Seven Failure Points When Engineering a Retrieval Augmented Generation System11 Jan

Other references · 11

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  3. 2016The Mythos of Model Interpretabilitycited by 716
  4. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 307
  5. 2019Requirements Engineering for Machine Learning: Perspectives from Data Scientistscited by 49
  6. 2020A Software Engineering Perspective on Engineering Machine Learning Systems: State of the Art and Challengescited by 33
  7. 2024RAG based Question-Answering for Contextual Response Prediction Systemcited by 14
  8. 2023Requirements Practices and Gaps When Engineering Human-Centered Artificial Intelligence Systemscited by 8
  9. 2024Transforming Software Development with Generative AI: Empirical Insights on Collaboration and Workflowcited by 6
  10. 2024Naming the Pain in Machine Learning-Enabled Systems Engineeringcited by 3
  11. 2024Teaching Theorizing in Software Engineering Researchcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Towards Energy-aware Requirements Dependency Classification: Knowledge-Graph vs. Vector-Retrieval Augmented Inference with SLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From Domain Documents to Requirements: Retrieval-Augmented Generation in the Space Industry
  2. 2025Retrieval-Augmented Generation in Industry: An Interview Study on Use Cases, Requirements, Challenges, and Evaluation
  3. 2024Generating Test Scenarios from NL Requirements using Retrieval-Augmented LLMs: An Industrial Study
  4. 2026An Industrial-Scale Retrieval-Augmented Generation Framework for Requirements Engineering: Empirical Evaluation with Automotive Manufacturing Data
  5. 2025Data Quality Challenges in Retrieval-Augmented Generation
  6. 2024RAG Does Not Work for Enterprises
  7. 2024Deploying Large Language Models With Retrieval Augmented Generation
  8. 2024Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems