# Introducing a new hyper-parameter for RAG: Context Window Utilization

arXiv: https://arxiv.org/abs/2407.19794 · HTML: https://mlsanity.com/paper/2407.19794/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 16 of 16)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Long-context LLMs Struggle with Long In-context Learning](https://mlsanity.com/paper/2404.02060/) (2024)
- [BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack](https://mlsanity.com/paper/2406.10149/) (2024)
- [Seven Failure Points When Engineering a Retrieval Augmented Generation System](https://mlsanity.com/paper/2401.05856/) (2024)
- [Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems](https://mlsanity.com/paper/2407.01370/) (2024)
- [Semantic Answer Similarity for Evaluating Question Answering Models](https://mlsanity.com/paper/2108.06130/) (2021)
- [LumberChunker: Long-Form Narrative Document Segmentation](https://mlsanity.com/paper/2406.17526/) (2024)
- [Exploring Dual Encoder Architectures for Question Answering](https://mlsanity.com/paper/2204.07120/) (2022)
- [Evaluating the Efficacy of Open-Source LLMs in Enterprise-Specific RAG Systems: A Comparative Study of Performance and Scalability](https://mlsanity.com/paper/2406.11424/) (2024)
- [Keyword Augmented Retrieval: Novel framework for Information Retrieval integrated with speech interface](https://mlsanity.com/paper/2310.04205/) (2023)
- [COS-Mix: Cosine Similarity and Distance Fusion for Improved Information Retrieval](https://mlsanity.com/paper/2406.00638/) (2024)
- [Benchmarking LLMs for Environmental Review and Permitting](https://mlsanity.com/paper/2407.07321/) (2024)

## Built on by (most cited first, 8 of 8)

- [A Survey on Large Language Models for Communication, Network, and Service Management: Application Insights, Challenges, and Future Directions](https://mlsanity.com/paper/2412.19823/) (2024)
- [Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoning](https://mlsanity.com/paper/2505.24478/) (2025)
- [KnowsLM: A framework for evaluation of small language models for knowledge augmentation and humanised conversations](https://mlsanity.com/paper/2504.04569/) (2025)
- [RAG Playground: A Framework for Systematic Evaluation of Retrieval Strategies and Prompt Engineering in RAG Systems](https://mlsanity.com/paper/2412.12322/) (2024)
- [UniC-RAG: Universal Knowledge Corruption Attacks to Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.18652/) (2025)
- [Prescriptive Agents based on RAG for Automated Maintenance (PARAM)](https://mlsanity.com/paper/2508.04714/) (2025)
- [i-LAVA: Insights on Low Latency Voice-2-Voice Architecture for Agents](https://mlsanity.com/paper/2509.20971/) (2025)
- [CultureVo: The Serious Game of Utilizing Gen AI for Enhancing Cultural Intelligence](https://mlsanity.com/paper/2407.20685/) (2024)

## Abstract (arXiv)

This paper introduces a new hyper-parameter for Retrieval-Augmented Generation (RAG) systems called Context Window Utilization. RAG systems enhance generative models by incorporating relevant information retrieved from external knowledge bases, improving the factual accuracy and contextual relevance of generated responses. The size of the text chunks retrieved and processed is a critical factor influencing RAG performance. This study aims to identify the optimal chunk size that maximizes answer generation quality. Through systematic experimentation, we analyze the effects of varying chunk sizes on the efficiency and effectiveness of RAG frameworks. Our findings reveal that an optimal chunk size balances the trade-off between providing sufficient context and minimizing irrelevant information. These insights are crucial for enhancing the design and implementation of RAG systems, underscoring the importance of selecting an appropriate chunk size to achieve superior performance.