mlsanityData to 7 Oct 2026

SEARA: An Automated Approach for Obtaining Optimal Retrievers

arXiv 2507.06554 ↗First posted 9 Jul 2025cs.IRcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) is a core approach for enhancing Large Language Models (LLMs), where the effectiveness of the retriever largely determines the overall response quality of RAG systems. Retrievers encompass a multitude of hyperparameters that significantly impact performance outcomes and demonstrate sensitivity to specific applications. Nevertheless, hyperparameter optimization entails prohibitively high computational expenses. Existing evaluation methods suffer from either prohibitive costs or disconnection from domain-specific scenarios. This paper proposes SEARA (Subset sampling Evaluation for Automatic Retriever Assessment), which addresses evaluation data challenges through subset sampling techniques and achieves robust automated retriever evaluation by minimal retrieval facts extraction and comprehensive retrieval metrics. Based on real user queries, this method enables fully automated retriever evaluation at low cost, thereby obtaining optimal retriever for specific business scenarios. We validate our method across classic RAG applications in rednote, including knowledge-based Q&A system and retrieval-based travel assistant, successfully obtaining scenario-specific optimal retrievers.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 9 of its 18 references that are in the map. First posted on arXiv 9 Jul 2025.

Builds on

9 of 18 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Evaluation of Retrieval-Augmented Generation: A Survey13 May
  2. 2024Blended RAG: Improving RAG (Retriever-Augmented Generation) Accuracy with Semantic Search and Hybrid Query-Based Retrievers22 Mar
  3. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries27 Jan
  4. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems16 Nov

Other references · 5

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  4. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  5. 2024Let LLMs Take on the Latest Challenges! A Chinese Dynamic Question Answering Benchmarkcited by 22

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Toward Optimal Search and Retrieval for RAG
  2. 2025Fast or Better? Balancing Accuracy and Cost in Retrieval-Augmented Generation with Flexible User Control
  3. 2025PRGB Benchmark: A Robust Placeholder-Assisted Algorithm for Benchmarking Retrieval-Augmented Generation
  4. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
  5. 2026Revisiting RAG Retrievers: An Information Theoretic Benchmark
  6. 2025QE-RAG: A Robust Retrieval-Augmented Generation Benchmark for Query Entry Errors
  7. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  8. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level