mlsanityData to 7 Oct 2026

Aligning Extraction and Generation for Robust Retrieval-Augmented Generation

arXiv 2503.04789 ↗First posted 28 Feb 2025cs.CL cs.AIcited by 2

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

also in 1 more
Abstract (arXiv)

Retrieval-augmented generation (RAG) enhances LLMs with external knowledge, yet generation remains vulnerable to retrieval-induced noise and uncertain placement of relevant chunks, often causing hallucinations. We present Ext2Gen, an extract-then-generate framework that strengthens LLMs via joint evidence selection and answer generation, dynamically identifying query-relevant content while suppressing noise, thereby removing the need for any independent pre-generation compression module. Optimized through preference alignment with well-curated pairwise feedback, Ext2Gen produces accurate and faithful answers even under noisy or imprecise retrieval. Experiments demonstrate that it substantially enhances the robustness of the generation backbone and yields greater performance gains than methods relying on independent compression models, e.g., Recomp, CompAct, EXIT). It further benefits from improved retrieval techniques such as query rewriting, underscoring that generation-side enhancements address limitations that retrieval alone cannot overcome.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 37 of its 56 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

37 of 56 references.

References · 37

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  7. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  8. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  9. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  10. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  11. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  12. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
25 more
  1. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  2. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
  3. 2021Efficient Attentions for Long Document Summarizationcited by 342
  4. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 292
  5. 2023Preference Ranking Optimization for Human Alignmentcited by 283
  6. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  7. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  8. 2024Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustmentcited by 171
  9. 2023Query2doc: Query Expansion with Large Language Modelscited by 146
  10. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  11. 2022Dense Text Retrieval based on Pretrained Language Models: A Surveycited by 136
  12. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 111
  13. 2024Reinforcement Learning for LLM Post-Training: A Surveycited by 91
  14. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  15. 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
  16. 2024Open-LLM-Leaderboard: From Multi-choice to Open-style Questions for LLMs Evaluation, Benchmark, and Arenacited by 76
  17. 2024CompAct: Compressing Retrieved Documents Actively for Question Answeringcited by 64
  18. 2024FIRST: Faster Improved Listwise Reranking with Single Token Decodingcited by 54
  19. 2021MS MARCO: Benchmarking Ranking Models in the Large-Data Regimecited by 28
  20. 2024Learning to Summarize from LLM-generated Feedbackcited by 24
  21. 2024Under the Surface: Tracking the Artifactuality of LLM-Generated Datacited by 23
  22. 2024EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generationcited by 17
  23. 2024Exploring the Best Practices of Query Expansion with Large Language Modelscited by 16
  24. 2024DSLR: Document Refinement with Sentence-Level Re-ranking and Reconstruction to Enhance Retrieval-Augmented Generationcited by 5
  25. 2024Progressive Query Expansion for Retrieval Over Cost-constrained Data Sourcescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Don't Scroll Back: Missing-Evidence Memory for Streaming Dialogue Summarizationcited by 0
  2. 2026CVE-SAI: Counterfactual Visual Evidence-Guided Selective Attribute Indexing for Risk-Controlled E-commerce Searchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  2. 2025Enhancing Retrieval-Augmented Generation with Two-Stage Retrieval: FlashRank Reranking and Query Expansion
  3. 2025RAGSmith: A Framework for Finding the Optimal Composition of Retrieval-Augmented Generation Methods Across Datasets
  4. 2025Investigating the Robustness of Retrieval-Augmented Generation at the Query Level
  5. 2026A Systems-Level Analysis of Sensitivity, Robustness, and Stability in Retrieval-Augmented Generation
  6. 2026OptiSet: Unified Optimizing Set Selection and Ranking for Retrieval-Augmented Generation
  7. 2026Less is More for RAG: Information Gain Pruning for Generator-Aligned Reranking and Evidence Selection
  8. 2025PRGB Benchmark: A Robust Placeholder-Assisted Algorithm for Benchmarking Retrieval-Augmented Generation