mlsanityData to 14 Oct 2026

Grounding by Trying: LLMs with Reinforcement Learning-Enhanced Retrieval

arXiv 2410.23214 ↗First posted 30 Oct 2024cs.LG cs.AIcited by 17

Not on the map. Why?

Abstract (arXiv)

The hallucinations of large language models (LLMs) are increasingly mitigated by allowing LLMs to search for information and to ground their answers in real sources. Unfortunately, LLMs often struggle with posing the right search queries, especially when dealing with complex or otherwise indirect topics. Observing that LLMs can learn to search for relevant facts by textittrying different queries and learning to up-weight queries that successfully produce relevant results, we introduce underlineLearning to underlineRetrieve by underlineTrying (LeReT), a reinforcement learning framework that explores search queries and uses preference-based optimization to improve their quality. LeReT can improve the absolute retrieval accuracy by up to 29% and the downstream generator evaluations by 17%. The simplicity and flexibility of LeReT allows it to be applied to arbitrary off-the-shelf retrievers and makes it a promising technique for improving general LLM pipelines. Project website: http://sherylhsu.com/LeReT/.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 32 of its 38 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

32 of 38 references.

References · 32

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2020Learning to summarize from human feedbackcited by 2,588
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  8. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  9. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  10. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  11. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  12. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
20 more
  1. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  2. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  3. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  4. 2023SLiC-HF: Sequence Likelihood Calibration with Human Feedbackcited by 401
  5. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
  6. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 291
  7. 2021Internet-Augmented Dialogue Generationcited by 217
  8. 2020HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verificationcited by 201
  9. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 197
  10. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 184
  11. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 162
  12. 2023Query2doc: Query Expansion with Large Language Modelscited by 149
  13. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  14. 2024Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithmscited by 143
  15. 2022Learning by Distilling Contextcited by 138
  16. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  17. 2021Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrievalcited by 63
  18. 2024Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Togethercited by 40
  19. 2017Task-Oriented Query Reformulation with Reinforcement Learningcited by 39
  20. 2023Copy Is All You Needcited by 27

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 6.

17 papers.

Sort:

By area:

  1. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  2. 2025Deep Research Agents: A Systematic Examination And Roadmapcited by 128
  3. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 62
  4. 2025Synergizing RAG and Reasoning: A Systematic Reviewcited by 27
  5. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  6. 2026Learning Next Action Predictors from Human-Computer Interactioncited by 7
  7. 2025Improving Context Fidelity via Native Retrieval-Augmented Reasoningcited by 5
  8. 2025FrugalRAG: Less is More in RL Finetuning for Multi-Hop Question Answeringcited by 4
  9. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Modelscited by 3
  10. 2026S3-R1: Learning to Retrieve and Answer Step-by-Step with Synthetic Datacited by 3
  11. 2025Beyond Text: Characterizing Domain Expert Needs in Document Researchcited by 1
  12. 2025Lucy: edgerunning agentic web search on mobile with machine generated task vectorscited by 1
5 more
  1. 2025Foundational Design Principles and Patterns for Building Robust and Adaptive GenAI-Native Systemscited by 1
  2. 2025QueStER: Query Specification for Generative keyword-based Retrievalcited by 1
  3. 2026MG2-RAG: Multi-Granularity Graph for Multimodal Retrieval-Augmented Generationcited by 1
  4. 2026Document Optimization for Black-Box Retrieval via Reinforcement Learningcited by 1
  5. 2026CoHyDE: Iterative Co-Training of LLM Rewriter & Dense Encoder for Tool Retrievalcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Lessons from Training Grounded LLMs with Verifiable Rewards
  2. 2025ReZero: Enhancing LLM search ability by trying one-more-time
  3. 2025SEM: Reinforcement Learning for Search-Efficient Large Language Models
  4. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searching
  5. 2025Rethinking On-policy Optimization for Query Augmentation
  6. 2025ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Reranking
  7. 2026Exploring LLM biases to manipulate AI search overview
  8. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning