mlsanityData to 7 Oct 2026

Improving Language Models via Plug-and-Play Retrieval Feedback

arXiv 2305.14002 ↗First posted 23 May 2023cs.CLcited by 87

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Large language models (LLMs) exhibit remarkable performance across various NLP tasks. However, they often generate incorrect or hallucinated information, which hinders their practical applicability in real-world scenarios. Human feedback has been shown to effectively enhance the factuality and quality of generated content, addressing some of these limitations. However, this approach is resource-intensive, involving manual input and supervision, which can be time-consuming and expensive. Moreover, it cannot be provided during inference, further limiting its practical utility in dynamic and interactive applications. In this paper, we introduce ReFeed, a novel pipeline designed to enhance LLMs by providing automatic retrieval feedback in a plug-and-play framework without the need for expensive fine-tuning. ReFeed first generates initial outputs, then utilizes a retrieval model to acquire relevant information from large document collections, and finally incorporates the retrieved information into the in-context demonstration for output refinement, thereby addressing the limitations of LLMs in a more efficient and cost-effective manner. Experiments on four knowledge-intensive benchmark datasets demonstrate our proposed ReFeed could improve over +6.0% under zero-shot setting and +2.5% under few-shot setting, compared to baselines without using retrieval feedback.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 34 of its 38 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

34 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Training Language Models with Language Feedback at Scale28 Mar
  2. 2022Training Language Models with Language Feedback29 Apr

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  11. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  12. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
20 more
  1. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  2. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  3. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  4. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  5. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  6. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  7. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  8. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  9. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  10. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  11. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  12. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  13. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  14. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  15. 2020A Survey of Knowledge-Enhanced Text Generationcited by 119
  16. 2021KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answeringcited by 66
  17. 2022Recitation-Augmented Language Modelscited by 65
  18. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  19. 2022Standing on the Shoulders of Giant Frozen Language Modelscited by 33
  20. 2022Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answeringcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 24; 2024: 38; 2025: 20; 2026: 5.

87 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  3. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  4. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  5. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  6. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  7. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  8. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  9. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
  10. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  11. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
  12. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
38 more
  1. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
  2. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  3. 2023Alignment for Honestycited by 72
  4. 2023Self-Knowledge Guided Retrieval Augmentation for Large Language Modelscited by 69
  5. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  6. 2023LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoningcited by 59
  7. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  8. 2023RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledgecited by 51
  9. 2024Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Modelscited by 47
  10. 2024Process-Driven Autoformalization in Lean 4cited by 45
  11. 2024In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigationcited by 43
  12. 2024Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?cited by 40
  13. 2024Large Language Models Can Self-Correct with Key Condition Verificationcited by 39
  14. 2023Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectificationcited by 38
  15. 2024Verification and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Provingcited by 35
  16. 2024Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMscited by 34
  17. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generationcited by 34
  18. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  19. 2024Self-DC: When to Reason and When to Act? Self Divide-and-Conquer for Compositional Unknown Questionscited by 30
  20. 2024MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMscited by 30
  21. 2024BlendFilter: Advancing Retrieval-Augmented Large Language Models via Query Generation Blending and Knowledge Filteringcited by 29
  22. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
  23. 2024EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimizationcited by 28
  24. 2024AutoPSV: Automated Process-Supervised Verifiercited by 28
  25. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
  26. 2023Towards Verifiable Text Generation with Evolving Memory and Self-Reflectioncited by 21
  27. 2025Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaborationcited by 19
  28. 2023Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinesecited by 18
  29. 2023Knowledge Plugins: Enhancing Large Language Models for Domain-Specific Recommendationscited by 16
  30. 2024Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuningcited by 16
  31. 2025R4ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systemscited by 16
  32. 2023ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?cited by 15
  33. 2024Learning From Correctness Without Prompting Makes LLM Efficient Reasonercited by 15
  34. 2024Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Enginescited by 14
  35. 2025Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewardscited by 14
  36. 2024FIT-RAG: Black-Box RAG with Factual Information and Token Reductioncited by 13
  37. 2023Adaptive-Solver Framework for Dynamic Strategy Selection in Large Language Model Reasoningcited by 12
  38. 2023TPE: Towards Better Compositional Reasoning over Conceptual Tools with Multi-persona Collaborationcited by 11

50 most cited of 87 All 87 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback
  2. 2024Think-then-Act: A Dual-Angle Evaluated Retrieval-Augmented Generation
  3. 2025Retrieval Enhanced Feedback via In-context Neural Error-book
  4. 2023Query Rewriting for Retrieval-Augmented Large Language Models
  5. 2026ReFilter: Improving Robustness of Retrieval-Augmented Generation via Gated Filter
  6. 2024FlashBack: Efficient Retrieval-Augmented Language Modeling for Fast Inference
  7. 2025DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation
  8. 2021Improving language models by retrieving from trillions of tokens