mlsanityData to 7 Oct 2026

SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs

arXiv 2404.13081 ↗First posted 17 Apr 2024cs.CL cs.AI cs.LGcited by 67

Not on the map. Why?

Cited by 47 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have made significant advancements in various natural language processing tasks, including question answering (QA) tasks. While incorporating new information with the retrieval of relevant passages is a promising way to improve QA with LLMs, the existing methods often require additional fine-tuning which becomes infeasible with recent LLMs. Augmenting retrieved passages via prompting has the potential to address this limitation, but this direction has been limitedly explored. To this end, we design a simple yet effective framework to enhance open-domain QA (ODQA) with LLMs, based on the summarized retrieval (SuRe). SuRe helps LLMs predict more accurate answers for a given question, which are well-supported by the summarized retrieval that could be viewed as an explicit rationale extracted from the retrieved passages. Specifically, SuRe first constructs summaries of the retrieved passages for each of the multiple answer candidates. Then, SuRe confirms the most plausible answer from the candidate set by evaluating the validity and ranking of the generated summaries. Experimental results on diverse ODQA benchmarks demonstrate the superiority of SuRe, with improvements of up to 4.6% in exact match (EM) and 4.0% in F1 score over standard prompting approaches. SuRe also can be integrated with a broad range of retrieval methods and LLMs. Finally, the generated summaries from SuRe show additional advantages to measure the importance of retrieved passages and serve as more preferred rationales by models and humans.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Builds on 61 of its 74 references that are in the map. First posted on arXiv 17 Apr 2024.

Builds on

61 of 74 references.

References · 61

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  10. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  12. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
38 more
  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  8. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  9. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  10. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  11. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  12. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  13. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  14. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  15. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  16. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  17. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  18. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  19. 2023Whose Opinions Do Language Models Reflect?cited by 759
  20. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  21. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  22. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  23. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  24. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  25. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  26. 2023Augmented Language Models: a Surveycited by 398
  27. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  28. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  29. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  30. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  31. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  32. 2022Prompting GPT-3 To Be Reliablecited by 258
  33. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  34. 2022A Contrastive Framework for Neural Text Generationcited by 224
  35. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 199
  36. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  37. 2020Adversarial Filters of Dataset Biasescited by 159
  38. 2023Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Stepcited by 144

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 40; 2026: 11.

67 papers.

Sort:

By area:

  1. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
  2. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  3. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  4. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 76
  5. 2025Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learningcited by 68
  6. 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
  7. 2025TrustRAG: Enhancing Robustness and Trustworthiness in Retrieval-Augmented Generationcited by 37
  8. 2025ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewardingcited by 32
  9. 2025R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learningcited by 32
  10. 2025R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learningcited by 30
  11. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
  12. 2025Removal of Hallucination on Hallucination: Debate-Augmented RAGcited by 29
38 more
  1. 2024TC-RAG:Turing-Complete RAG's Case study on Medical LLM Systemscited by 28
  2. 2024Retrieval-augmented generation in multilingual settingscited by 27
  3. 2024Does RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation Systemscited by 26
  4. 2025Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Rewardcited by 25
  5. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 17
  6. 2025RAS: Retrieval-And-Structuring for Knowledge-Intensive LLM Generationcited by 16
  7. 2024Retrieve-Plan-Generation: An Iterative Planning and Answering Framework for Knowledge-Intensive LLM Generationcited by 15
  8. 2025GainRAG: Preference Alignment in Retrieval-Augmented Generation through Gain Signal Synthesiscited by 12
  9. 2025Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generationcited by 12
  10. 2025SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAGcited by 12
  11. 2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
  12. 2025ComposeRAG: A Modular and Composable RAG for Corpus-Grounded Multi-Hop Question Answeringcited by 10
  13. 2024SEER: Self-Aligned Evidence Extraction for Retrieval-Augmented Generationcited by 9
  14. 2025Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAGcited by 9
  15. 2025SuperRAG: Beyond RAG with Layout-Aware Graph Modelingcited by 9
  16. 2025ReZero: Enhancing LLM search ability by trying one-more-timecited by 9
  17. 2025EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notescited by 8
  18. 2025Toward Structured Knowledge Reasoning: Contrastive Retrieval-Augmented Generation on Experiencecited by 7
  19. 2025FAIR-RAG: Faithful Adaptive Iterative Refinement for Retrieval-Augmented Generationcited by 7
  20. 2024ProveRAG: Provenance-Driven Vulnerability Analysis with Automated Retrieval-Augmented LLMscited by 6
  21. 2025Optimizing Question Semantic Space for Dynamic Retrieval-Augmented Multi-hop Question Answeringcited by 6
  22. 2026GRACE: Reinforcement Learning for Grounded Response and Abstention under Contextual Evidencecited by 6
  23. 2025Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotationcited by 5
  24. 2024GRS-QA -- Graph Reasoning-Structured Question Answering Datasetcited by 4
  25. 2025Training-free LLM Verification via Recycling Few-shot Examplescited by 4
  26. 2025CoCoA: Collaborative Chain-of-Agents for Parametric-Retrieved Knowledge Synergycited by 4
  27. 2024Dynamic Uncertainty Ranking: Enhancing Retrieval-Augmented In-Context Learning for Long-Tail Knowledge in LLMscited by 3
  28. 2024MIRe: Enhancing Multimodal Queries Representation via Fusion-Free Modality Interaction for Multimodal Retrievalcited by 3
  29. 2025PAIRS: Parametric-Verified Adaptive Information Retrieval and Selection for Efficient RAGcited by 3
  30. 2026Decide Then Retrieve: A Training-Free Framework with Uncertainty-Guided Triggering and Dual-Path Retrievalcited by 3
  31. 2026OptiSet: Unified Optimizing Set Selection and Ranking for Retrieval-Augmented Generationcited by 3
  32. 2024Towards Human-Level Understanding of Complex Process Engineering Schematics: A Pedagogical, Introspective Multi-Agent Framework for Open-Domain Question Answeringcited by 2
  33. 2025K-COMP: Retrieval-Augmented Medical Domain Question Answering With Knowledge-Injected Compressorcited by 2
  34. 2025Optimizing Knowledge Integration in Retrieval-Augmented Generation with Self-Selectioncited by 2
  35. 2025LLM Embedding-based Attribution (LEA): Quantifying Source Contributions to Generative Model's Response for Vulnerability Analysiscited by 2
  36. 2025MuISQA: Multi-Intent Retrieval-Augmented Generation for Scientific Question Answeringcited by 2
  37. 2025R4: Retrieval-Augmented Reasoning for Vision-Language Models in 4D Spatio-Temporal Spacecited by 2
  38. 2026HALT: Verification-Aware Stopping for Retrieval-Augmented Search Agentscited by 2

50 most cited of 67 All 67 (JSON)