mlsanityData to 14 Oct 2026

Retrieval-Augmented Generation for Domain-Specific Question Answering: A Case Study on Pittsburgh and CMU

arXiv 2411.13691 ↗First posted 20 Nov 2024cs.LG cs.CLcited by 1

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

We designed a Retrieval-Augmented Generation (RAG) system to provide large language models with relevant documents for answering domain-specific questions about Pittsburgh and Carnegie Mellon University (CMU). We extracted over 1,800 subpages using a greedy scraping strategy and employed a hybrid annotation process, combining manual and Mistral-generated question-answer pairs, achieving an inter-annotator agreement (IAA) score of 0.7625. Our RAG framework integrates BM25 and FAISS retrievers, enhanced with a reranker for improved document retrieval accuracy. Experimental results show that the RAG system significantly outperforms a non-RAG baseline, particularly in time-sensitive and complex queries, with an F1 score improvement from 5.45% to 42.21% and recall of 56.18%. This study demonstrates the potential of RAG systems in enhancing answer precision and relevance, while identifying areas for further optimization in document retrieval and model training.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 2 of its 5 references that are in the map. First posted on arXiv 20 Nov 2024.

Builds on

2 of 5 references.

References · 2

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  2. 2023Mistral 7Bcited by 3,164

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Your RAG is Unfair: Exposing Fairness Vulnerabilities in Retrieval-Augmented Generation via Backdoor Attackscited by 8

Similar, not cited

Close in wording, no citation either way

  1. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions
  2. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
  3. 2025QuIM-RAG: Advancing Retrieval-Augmented Generation with Inverted Question Matching for Enhanced QA Performance
  4. 2025RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learning
  5. 2025Domain-Specific Data Generation Framework for RAG Adaptation
  6. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generation
  7. 2024DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answering
  8. 2024Towards Optimizing a Retrieval Augmented Generation using Large Language Model on Academic Data