mlsanityData to 14 Oct 2026

METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation

arXiv 2412.10543 ↗First posted 13 Dec 2024cs.LG cs.CL cs.IRcited by 17

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

RAG (Retrieval Augmented Generation) allows LLMs (large language models) to generate better responses with external knowledge, but using more external knowledge often improves generation quality at the expense of response delay. Prior work either reduces the response delay (through better scheduling of RAG queries) or strives to maximize quality (which involves tuning the RAG workflow), but they fall short in optimizing the tradeoff between the delay and quality of RAG responses. This paper presents METIS, the first RAG system that jointly schedules queries and adapts the key RAG configurations of each query, such as the number of retrieved text chunks and synthesis methods, in order to balance quality optimization and response delay reduction. Using 4 popular RAG-QA datasets, we show that compared with the state-of-the-art RAG optimization schemes, METIS reduces the generation latency by 1.64-2.54× without sacrificing generation quality.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 55 of its 87 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

55 of 87 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024AutoRAG: Automated Framework for optimization of Retrieval Augmented Generation Pipeline28 Oct
  2. 2024A Methodology for Evaluating RAG Systems: A Case Study On Configuration Dependency Validation11 Oct
  3. 2024RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework2 Aug
  4. 2024AutoRAG-HP: Automatic Online Hyper-Parameter Tuning for Retrieval-Augmented Generation27 Jun
  5. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models10 May
  6. 2024PipeRAG: Fast Retrieval-Augmented Generation via Algorithm-System Co-design8 Mar
  7. 2024The Power of Noise: Redefining Retrieval for RAG Systems26 Jan

Other references · 48

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  7. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  8. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  9. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  10. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  11. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  12. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
36 more
  1. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 365
  2. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 361
  3. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  4. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 262
  5. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
  6. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
  7. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  8. 2023PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUcited by 217
  9. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
  10. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 203
  11. 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentioncited by 163
  12. 2024MiniCache: KV Cache Compression in Depth Dimension for Large Language Modelscited by 125
  13. 2024LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelismcited by 112
  14. 2024Parrot: Efficient Serving of LLM-based Applications with Semantic Variablecited by 99
  15. 2024Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Taskscited by 96
  16. 2024PQCache: Product Quantization-based KVCache for Long Context LLM Inferencecited by 95
  17. 2024NanoFlow: Towards Optimal Large Language Model Serving Throughputcited by 90
  18. 2024Searching for Best Practices in Retrieval-Augmented Generationcited by 83
  19. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 82
  20. 2024RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculturecited by 81
  21. 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inferencecited by 75
  22. 2024Preble: Efficient Distributed Prompt Scheduling for LLM Servingcited by 70
  23. 2024MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Poolcited by 68
  24. 2023Stateful Large Language Model Serving with Pensievecited by 61
  25. 2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 56
  26. 2024MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Modelscited by 42
  27. 2024NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inferencecited by 37
  28. 2024Routoo: Learning to Route to Large Language Models Effectivelycited by 34
  29. 2024InferCept: Efficient Intercept Support for Augmented Large Language Model Inferencecited by 33
  30. 2024Marconi: Prefix Caching for the Era of Hybrid LLMscited by 31
  31. 2024MBA-RAG: a Bandit Approach for Adaptive Retrieval-Augmented Generation through Question Complexitycited by 29
  32. 2024Long Context RAG Performance of Large Language Modelscited by 23
  33. 2024Do Large Language Models Need a Content Delivery Network?cited by 16
  34. 2023Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Servingcited by 15
  35. 2024Fast Inference for Augmented Large Language Modelscited by 10
  36. 2022Long Document Re-ranking with Modular Re-rankercited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 15.

17 papers.

Sort:

By area:

  1. 2025EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Servingcited by 10
  2. 2026Pancake: Hierarchical Memory System for Multi-Agent LLM Servingcited by 5
  3. 2026An experimental study of KV cache reuse strategies in chunk-level caching systemscited by 5
  4. 2026PCR: A Prefetch-Enhanced Cache Reuse System for Low-Latency RAG Servingcited by 4
  5. 2026PEEK: Context Map as an Orientation Cache for Long-Context LLM Agentscited by 4
  6. 2025Practical Code RAG at Scale: Task-Aware Retrieval Design Choices under Compute Budgetscited by 3
  7. 2026Can QPP Choose the Right Query Variant? Evaluating Query Variant Selection for RAG Pipelinescited by 1
  8. 2026CDS4RAG: Cyclic Dual-Sequential Hyperparameter Optimization for RAGcited by 1
  9. 2026Diagnosing and Repairing Factual Errors in RAG under Budget Constraintscited by 1
  10. 2026RAG-Stack: Co-Optimizing RAG Serving Performance and Qualitycited by 1
  11. 2026Less can be More: Relieving RAG Bottlenecks via Evidence Frontloading and Pressure-Adaptive Budgetingcited by 1
  12. 2026Natural Language Query to Configuration for Retrieval Agentscited by 0
5 more
  1. 2026Enhancing SLMs for Sustainable Code Optimization in Radio-Astronomycited by 0
  2. 2026From Retrieved Context to Runtime Control: Adaptive Compression for Edge-based RAGcited by 0
  3. 2026Atlas: Optimizing Deployment of Compound AI Workflows on Heterogeneous Clusterscited by 0
  4. 2026Evaluating the accuracy of KV cache reuse techniquescited by 0
  5. 2026Configuring RAG Systems: What Breaks, Where Originates, and How to Fixcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving
  2. 2025RAG-Stack: Co-Optimizing RAG Quality and Performance From the Vector Database Perspective
  3. 2024RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation
  4. 2024Toward Optimal Search and Retrieval for RAG
  5. 2025Harmonia: End-to-End RAG Serving Optimization
  6. 2024XRAG: eXamining the Core -- Benchmarking Foundational Components in Advanced Retrieval-Augmented Generation
  7. 2024Retrieval Augmented Generation Systems: Automatic Dataset Creation, Evaluation and Boolean Agent Setup
  8. 2026RAISE: RAG Design as an Architecture Search Problem