mlsanityData to 7 Oct 2026

Towards Optimizing a Retrieval Augmented Generation using Large Language Model on Academic Data

arXiv 2411.08438 ↗First posted 13 Nov 2024cs.AIcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Given the growing trend of many organizations integrating Retrieval Augmented Generation (RAG) into their operations, we assess RAG on domain-specific data and test state-of-the-art models across various optimization techniques. We incorporate four optimizations; Multi-Query, Child-Parent-Retriever, Ensemble Retriever, and In-Context-Learning, to enhance the functionality and performance in the academic domain. We focus on data retrieval, specifically targeting various study programs at a large technical university. We additionally introduce a novel evaluation approach, the RAG Confusion Matrix designed to assess the effectiveness of various configurations within the RAG framework. By exploring the integration of both open-source (e.g., Llama2, Mistral) and closed-source (GPT-3.5 and GPT-4) Large Language Models, we offer valuable insights into the application and optimization of RAG frameworks in domain-specific contexts. Our experiments show a significant performance increase when including multi-query in the retrieval phase.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 16 of its 31 references that are in the map. First posted on arXiv 13 Nov 2024.

Builds on

16 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Towards Optimizing and Evaluating a Retrieval Augmented QA Chatbot using LLMs with Human in the Loop8 Jul
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec

Other references · 14

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  5. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  6. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  7. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  8. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  9. 2022Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answeringcited by 126
  10. 2022Re2G: Retrieve, Rerank, Generatecited by 102
  11. 2021Neural Machine Translation with Monolingual Translation Memorycited by 61
  12. 2023Empower Large Language Model to Perform Better on Industrial Domain-Specific Question Answeringcited by 41
2 more
  1. 2020Controlling Dialogue Generation with Semantic Exemplarscited by 25
  2. 2021RetrievalSum: A Retrieval Enhanced Framework for Abstractive Summarizationcited by 19

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Enhancing Retrieval-Augmented Generation: A Study of Best Practices
  2. 2024RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation
  3. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
  4. 2024CoFE-RAG: A Comprehensive Full-chain Evaluation Framework for Retrieval-Augmented Generation with Enhanced Data Diversity
  5. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
  6. 2025Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Control
  7. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  8. 2025When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs