mlsanityData to 7 Oct 2026

ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation

arXiv 2502.08178 ↗First posted 12 Feb 2025cs.CLcited by 1

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

While Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by incorporating external knowledge, they still face persistent challenges in retrieval inefficiency and the inability of LLMs to filter out irrelevant information. We present ParetoRAG, an unsupervised framework that optimizes RAG systems through sentence-level refinement guided by the Pareto principle. By decomposing paragraphs into sentences and dynamically re-weighting core content while preserving contextual coherence, ParetoRAG achieves dual improvements in both retrieval precision and generation quality without requiring additional training or API resources. This framework has been empirically validated across various datasets, LLMs, and retrievers.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 34 of its 45 references that are in the map. First posted on arXiv 12 Feb 2025.

Builds on

34 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG8 Oct

Other references · 33

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  12. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
21 more
  1. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  2. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  4. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  5. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  6. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  7. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  8. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  9. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  10. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  11. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  12. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  13. 2022RealTime QA: What's the Answer Right Now?cited by 239
  14. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  15. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
  16. 2021Decontextualization: Making Sentences Stand-Alonecited by 101
  17. 2024Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Trainingcited by 88
  18. 2024SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMscited by 67
  19. 2024RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedbackcited by 46
  20. 2023ALCUNA: Large Language Models Meet New Knowledgecited by 35
  21. 2021Phrase Retrieval Learns Passage Retrieval, Toocited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Retrieval-Augmented Detection of Potentially Abusive Clauses in Chilean Terms of Servicecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Enhancing Retrieval-Augmented Generation: A Study of Best Practices
  2. 2025Dynamic Context Selection for Retrieval-Augmented Generation: Mitigating Distractors and Positional Bias
  3. 2025Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter Impact on Performance and Efficiency
  4. 2024Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Survey
  5. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  6. 2025ImpRAG: Retrieval-Augmented Generation with Implicit Queries
  7. 2025Grounding Long-Context Reasoning with Contextual Normalization for Retrieval-Augmented Generation
  8. 2025Oreo: A Plug-in Context Reconstructor to Enhance Retrieval-Augmented Generation