mlsanityData to 7 Oct 2026

Estimating Optimal Context Length for Hybrid Retrieval-augmented Multi-document Summarization

arXiv 2504.12972 ↗First posted 17 Apr 2025cs.CLcited by 1

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Recent advances in long-context reasoning abilities of language models led to interesting applications in large-scale multi-document summarization. However, prior work has shown that these long-context models are not effective at their claimed context windows. To this end, retrieval-augmented systems provide an efficient and effective alternative. However, their performance can be highly sensitive to the choice of retrieval context length. In this work, we present a hybrid method that combines retrieval-augmented systems with long-context windows supported by recent language models. Our method first estimates the optimal retrieval length as a function of the retriever, summarizer, and dataset. On a randomly sampled subset of the dataset, we use a panel of LLMs to generate a pool of silver references. We use these silver references to estimate the optimal context length for a given RAG system configuration. Our results on the multi-document summarization task showcase the effectiveness of our method across model classes and sizes. We compare against length estimates from strong long-context benchmarks such as RULER and HELMET. Our analysis also highlights the effectiveness of our estimation method for very long-context LMs and its generalization to new classes of LMs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 23 references that are in the map. First posted on arXiv 17 Apr 2025.

Builds on

18 of 23 references.

References · 18

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  3. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  4. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  5. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
  6. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
  7. 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 141
  8. 2024How to Train Long-Context Language Models (Effectively)cited by 141
  9. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAGcited by 122
  10. 2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approachcited by 117
  11. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
  12. 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
6 more
  1. 2024One Thousand and One Pairs: A "novel" challenge for long-context language modelscited by 80
  2. 2022Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decodingcited by 54
  3. 2024In Defense of RAG in the Era of Long-Context Language Modelscited by 52
  4. 2023It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Riskcited by 49
  5. 2023Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluationcited by 24
  6. 2025Scaling Multi-Document Event Summarization: Evaluating Compression vs. Full-Text Approachescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs
  2. 2025Considering Length Diversity in Retrieval-Augmented Summarization
  3. 2024Long Context RAG Performance of Large Language Models
  4. 2025More Documents, Same Length: Isolating the Challenge of Multiple Documents in RAG
  5. 2025MacRAG: Compress, Slice, and Scale-up for Multi-Scale Adaptive Context RAG
  6. 2025Grounding Long-Context Reasoning with Contextual Normalization for Retrieval-Augmented Generation
  7. 2025Stronger Baselines for Retrieval-Augmented Generation with Long-Context Language Models
  8. 2024Towards a Robust Retrieval-Based Summarization System