mlsanityData to 7 Oct 2026

CDS4RAG: Cyclic Dual-Sequential Hyperparameter Optimization for RAG

arXiv 2605.08333 ↗First posted 8 May 2026cs.LG cs.AI cs.CLcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) is sensitive to the vast hyperparameters of the retriever and generator, yet optimizing them using given queries is a challenging task due to the complex interactions and expensive evaluation costs. Existing algorithms are ineffective and slow in convergence, since they often treat RAG as a monolithic black box or only optimize partial hyperparameters. In this paper, we propose CDS4RAG, a framework that optimizes the full RAG hyperparameters using given queries via a new cyclic dual-sequential formulation. CDS4RAG is special in the sense that it distinguishes the hyperparameters of the retriever and generator, cyclically optimizing them in turn. Such a paradigm allows us to design fine-grained within-cycle budget provision and expedite the optimization via cross-cycle seeding when optimizing the generator. CDS4RAG is also an algorithm-agnostic framework that can be paired with diverse general algorithms. Through experiments on four common benchmarks and two backbone LLMs, we reveal that CDS4RAG considerably boosts the vanilla algorithms in 21/24 cases while significantly outperforming state-of-the-art algorithms in all cases with up to 1.54x improvements of generation quality and better speedup.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 37 of its 45 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

37 of 45 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025An Analysis of Hyper-Parameter Optimization Methods for Retrieval Augmented Generation6 May
  2. 2025Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systems25 Feb
  3. 2024METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation13 Dec
  4. 2024AutoRAG-HP: Automatic Online Hyper-Parameter Tuning for Retrieval-Augmented Generation27 Jun
  5. 2024Evaluation of Retrieval-Augmented Generation: A Survey13 May
  6. 2024Evaluating Retrieval Quality in Retrieval-Augmented Generation21 Apr

Other references · 31

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
  7. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
  8. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  9. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  10. 2024LightRAG: Simple and Fast Retrieval-Augmented Generationcited by 310
  11. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
  12. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
19 more
  1. 2017A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applicationscited by 126
  2. 2020HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisationcited by 81
  3. 2020AutoML to Date and Beyond: Challenges and Opportunitiescited by 63
  4. 2021Multi-Objectivizing Software Configuration Tuning (for a single performance concern)cited by 27
  5. 2021MMO: Meta Multi-Objectivization for Software Configuration Tuningcited by 19
  6. 2024Predicting Configuration Performance in Multiple Environments with Sequential Meta-learningcited by 17
  7. 2022Lifelong Dynamic Optimization for Self-Adaptive Systems: Fact or Fiction?cited by 16
  8. 2023Predicting Software Performance with Divide-and-Learncited by 16
  9. 2024Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworkscited by 14
  10. 2024Adapting Multi-objectivized Software Configuration Tuningcited by 13
  11. 2025Distilled Lifelong Self-Adaptation for Configurable Systemscited by 11
  12. 2025PromiseTune: Unveiling Causally Promising and Explainable Configuration Tuningcited by 11
  13. 2023OpenBox: A Python Toolkit for Generalized Black-box Optimizationcited by 9
  14. 2025Accuracy Can Lie: On the Impact of Surrogate Model in Configuration Tuningcited by 9
  15. 2025Dually Hierarchical Drift Adaptation for Online Configuration Performance Learningcited by 7
  16. 2024Dividable Configuration Performance Learningcited by 4
  17. 2026Revealing Domain-Spatiality Patterns for Configuration Tuning: Domain Knowledge Meets Fitness Landscapescited by 4
  18. 2025AI4Contracts: LLM & RAG-Powered Encoding of Financial Derivative Contractscited by 2
  19. 2025CoTune: Co-evolutionary Configuration Tuningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving
  2. 2025RAG-Stack: Co-Optimizing RAG Quality and Performance From the Vector Database Perspective
  3. 2026RAISE: RAG Design as an Architecture Search Problem
  4. 2024AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline
  5. 2025FB-RAG: Improving RAG with Forward and Backward Lookup
  6. 2026RAG-Stack: Co-Optimizing RAG Serving Performance and Quality
  7. 2025HedraRAG: Coordinating LLM Generation and Database Retrieval in Heterogeneous RAG Serving
  8. 2025DRAGON: Domain-specific Robust Automatic Data Generation for RAG Optimization