mlsanityData to 14 Oct 2026

Towards Lighter and Robust Evaluation for Retrieval Augmented Generation

arXiv 2503.16161 ↗First posted 20 Mar 2025cs.CL cs.AIcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Large Language Models are prompting us to view more NLP tasks from a generative perspective. At the same time, they offer a new way of accessing information, mainly through the RAG framework. While there have been notable improvements for the autoregressive models, overcoming hallucination in the generated answers remains a continuous problem. A standard solution is to use commercial LLMs, such as GPT4, to evaluate these algorithms. However, such frameworks are expensive and not very transparent. Therefore, we propose a study which demonstrates the interest of open-weight models for evaluating RAG hallucination. We develop a lightweight approach using smaller, quantized LLMs to provide an accessible and interpretable metric that gives continuous scores for the generated answer with respect to their correctness and faithfulness. This score allows us to question decisions' reliability and explore thresholds to develop a new AUC metric as an alternative to correlation with human judgment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 19 of its 23 references that are in the map. First posted on arXiv 20 Mar 2025.

Builds on

19 of 23 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation26 Sep

Other references · 18

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  8. 2024Mixtral of Expertscited by 1,823
  9. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  10. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  11. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  12. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 270
6 more
  1. 2023Efficient Guided Generation for Large Language Modelscited by 200
  2. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 151
  3. 2021TopiOCQA: Open-domain Conversational Question Answering with Topic Switchingcited by 96
  4. 2024GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Frameworkcited by 30
  5. 2023Interpretable Unified Language Checkingcited by 26
  6. 2020Natural Language Inference in Context -- Investigating Contextual Reasoning over Long Textscited by 23

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Evaluation of Retrieval-Augmented Generation: A Survey
  2. 2024The Power of Noise: Redefining Retrieval for RAG Systems
  3. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
  4. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  5. 2025Finetune-RAG: Fine-Tuning Language Models to Resist Hallucination in Retrieval-Augmented Generation
  6. 2026Attribution Techniques for Mitigating Hallucinated Information in RAG Systems: A Survey
  7. 2025RAG-Check: Evaluating Multimodal Retrieval Augmented Generation Performance
  8. 2025Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards