mlsanityData to 7 Oct 2026

Re2G: Retrieve, Rerank, Generate

arXiv 2207.06300 ↗First posted 13 Jul 2022cs.CL cs.AI cs.IRcited by 102

Not on the map. Why?

Cited by 54 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

As demonstrated by GPT-3 and T5, transformers grow in capability as parameter spaces become larger and larger. However, for tasks that require a large amount of knowledge, non-parametric memory allows models to grow dramatically with a sub-linear increase in computational cost and GPU memory requirements. Recent models such as RAG and REALM have introduced retrieval into conditional generation. These models incorporate neural initial retrieval from a corpus of passages. We build on this line of research, proposing Re2G, which combines both neural initial retrieval and reranking into a BART-based sequence-to-sequence generation. Our reranking approach also permits merging retrieval results from sources with incomparable scores, enabling an ensemble of BM25 and neural initial retrieval. To train our system end-to-end, we introduce a novel variation of knowledge distillation to train the initial retrieval, reranker, and generation using only ground truth on the target sequence output. We find large gains in four diverse tasks: zero-shot slot filling, question answering, fact-checking, and dialog, with relative gains of 9% to 34% over the previous state-of-the-art on the KILT leaderboard. We make our code available as open source at https://github.com/IBM/kgi-slot-filling/tree/re2g.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). Builds on 29 of its 55 references that are in the map. First posted on arXiv 13 Jul 2022.

Builds on

29 of 55 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2017Billion-scale similarity search with GPUscited by 2,477
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  12. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
17 more
  1. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  2. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  3. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  4. 2019Passage Re-ranking with BERTcited by 629
  5. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  6. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  7. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  8. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  9. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  10. 2020Autoregressive Entity Retrievalcited by 333
  11. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 112
  12. 2018The Fact Extraction and VERification (FEVER) Shared Taskcited by 108
  13. 2020Learning Dense Representations of Phrases at Scalecited by 91
  14. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  15. 2021Multi-task Retrieval for Knowledge-Intensive Taskscited by 44
  16. 2021Hindsight: Posterior-guided training of retrievers for improved open-ended generationcited by 38
  17. 2021Robust Retrieval Augmented Generation for Zero-shot Slot Fillingcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 18; 2024: 26; 2025: 40; 2026: 13.

102 papers.

Sort:

By area:

  1. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  2. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  3. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  4. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graphcited by 235
  5. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  6. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  7. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  8. 2022FiD-Light: Efficient and Effective Retrieval-Augmented Text Generationcited by 63
  9. 2024Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Modelscited by 57
  10. 2024Don't Forget to Connect! Improving RAG with Graph-based Rerankingcited by 49
  11. 2025When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generationcited by 46
  12. 2024RAG-Fusion: a New Take on Retrieval-Augmented Generationcited by 42
38 more
  1. 2025TrustRAG: Enhancing Robustness and Trustworthiness in Retrieval-Augmented Generationcited by 37
  2. 2023WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpuscited by 29
  3. 2024Jill Watson: A Virtual Teaching Assistant powered by ChatGPTcited by 24
  4. 2024Claim Verification in the Age of Large Language Models: A Surveycited by 23
  5. 2023Unified Language Representation for Question Answering over Text, Tables, and Imagescited by 22
  6. 2024SmartRAG: Jointly Learn RAG-Related Tasks From the Environment Feedbackcited by 19
  7. 2025Benchmarking Retrieval-Augmented Generation for Chemistrycited by 19
  8. 2025Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queriescited by 19
  9. 2025RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Servingcited by 18
  10. 2025EnrichIndex: Using LLMs to Enrich Retrieval Indices Offlinecited by 15
  11. 2024Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Factscited by 14
  12. 2025Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systemscited by 14
  13. 2022KnowGL: Knowledge Generation and Linking from Textcited by 13
  14. 2024VerMCTS: Synthesizing Multi-Step Programs using a Verifier, a Large Language Model, and Tree Searchcited by 13
  15. 2023RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modelingcited by 12
  16. 2024Retrieving, Rethinking and Revising: The Chain-of-Verification Can Improve Retrieval Augmented Generationcited by 12
  17. 2023HiPrompt: Few-Shot Biomedical Knowledge Fusion via Hierarchy-Oriented Promptingcited by 11
  18. 2023GripRank: Bridging the Gap between Retrieval and Generation via the Generative Knowledge Improved Passage Rankingcited by 11
  19. 2023Unleashing Potential of Evidence in Knowledge-Intensive Dialogue Generationcited by 11
  20. 2024No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Userscited by 9
  21. 2025Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generationcited by 9
  22. 2024A Hybrid RAG System with Comprehensive Enhancement on Complex Reasoningcited by 8
  23. 2024Drowning in Documents: Consequences of Scaling Reranker Inferencecited by 8
  24. 2025Retrieval-Augmented Generation in Industry: An Interview Study on Use Cases, Requirements, Challenges, and Evaluationcited by 8
  25. 2025EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notescited by 8
  26. 2025H2R: Hierarchical Hindsight Reflection for Multi-Task LLM Agentscited by 8
  27. 2025Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalizationcited by 8
  28. 2024Reward-RAG: Enhancing RAG with Reward Driven Supervisioncited by 7
  29. 2026Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimizationcited by 7
  30. 2022Adaptation Approaches for Nearest Neighbor Language Modelscited by 6
  31. 2023Coarse-to-Fine Knowledge Selection for Document Grounded Dialogscited by 6
  32. 2025Automatic Dataset Generation for Knowledge Intensive Question Answering Taskscited by 6
  33. 2025Automatic Prompt Optimization for Knowledge Graph Construction: Insights from an Empirical Studycited by 6
  34. 2025HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generationcited by 6
  35. 2023Progressive Evidence Refinement for Open-domain Multimodal Retrieval Question Answeringcited by 5
  36. 2024HiQA: A Hierarchical Contextual Augmentation RAG for Multi-Documents QAcited by 5
  37. 2025Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generationcited by 5
  38. 2025Demystifying and Enhancing the Efficiency of Large Language Model Based Search Agentscited by 5

50 most cited of 102 All 102 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Rank-K: Test-Time Reasoning for Listwise Reranking
  2. 2025Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challenge
  3. 2025Rank1: Test-Time Compute for Reranking in Information Retrieval
  4. 2025DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation
  5. 2024Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers
  6. 2025ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Reranking
  7. 2025Enhancing Retrieval-Augmented Generation with Two-Stage Retrieval: FlashRank Reranking and Query Expansion
  8. 2026SciRet: A Compute-Aware Empirical Study of Retrieval and Reranking for Scientific RAG