# LLatrieval: LLM-Verified Retrieval for Verifiable Generation

arXiv: https://arxiv.org/abs/2311.07838 · HTML: https://mlsanity.com/paper/2311.07838/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 65 references that are in the map.
- First posted on arXiv 14 Nov 2023.

## Builds on (most cited first, 30 of 46)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Making Retrieval-Augmented Language Models Robust to Irrelevant Context](https://mlsanity.com/paper/2310.01558/) (2023)
- [A Survey of Hallucination in Large Foundation Models](https://mlsanity.com/paper/2309.05922/) (2023)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://mlsanity.com/paper/2305.15294/) (2023)
- [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://mlsanity.com/paper/2212.09741/) (2022)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [Teaching language models to support answers with verified quotes](https://mlsanity.com/paper/2203.11147/) (2022)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://mlsanity.com/paper/2210.08726/) (2022)
- [Measuring Attribution in Natural Language Generation Models](https://mlsanity.com/paper/2112.12870/) (2021)
- [FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios](https://mlsanity.com/paper/2307.13528/) (2023)
- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://mlsanity.com/paper/2209.11755/) (2022)
- [ASQA: Factoid Questions Meet Long-Form Answers](https://mlsanity.com/paper/2204.06092/) (2022)

## Built on by (most cited first, 30 of 52)

- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach](https://mlsanity.com/paper/2407.16833/) (2024)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [Trustworthiness in Retrieval-Augmented Generation Systems: A Survey](https://mlsanity.com/paper/2409.10102/) (2024)
- [MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation](https://mlsanity.com/paper/2501.00332/) (2024)
- [BlendFilter: Advancing Retrieval-Augmented Large Language Models via Query Generation Blending and Knowledge Filtering](https://mlsanity.com/paper/2402.11129/) (2024)
- [Improving Attributed Text Generation of Large Language Models via Preference Learning](https://mlsanity.com/paper/2403.18381/) (2024)
- [AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attribution](https://mlsanity.com/paper/2411.15102/) (2024)
- [Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generation](https://mlsanity.com/paper/2407.01796/) (2024)
- [Atomic Fact Decomposition Helps Attributed Question Answering](https://mlsanity.com/paper/2410.16708/) (2024)
- [Reliable Text-to-SQL with Adaptive Abstention](https://mlsanity.com/paper/2501.10858/) (2025)
- [Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics](https://mlsanity.com/paper/2406.15264/) (2024)
- [Verifiable Generation with Subsentence-Level Fine-Grained Citations](https://mlsanity.com/paper/2406.06125/) (2024)
- [TokenShapley: Token Level Context Attribution with Shapley Value](https://mlsanity.com/paper/2507.05261/) (2025)
- [VerifyLLM: LLM-Based Pre-Execution Task Plan Verification for Robots](https://mlsanity.com/paper/2507.05118/) (2025)
- [CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation](https://mlsanity.com/paper/2406.05365/) (2024)
- [Diversify-verify-adapt: Efficient and Robust Retrieval-Augmented Ambiguous Question Answering](https://mlsanity.com/paper/2409.02361/) (2024)
- [RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization](https://mlsanity.com/paper/2502.10993/) (2025)
- [RAC3: Retrieval-Augmented Corner Case Comprehension for Autonomous Driving with Vision-Language Models](https://mlsanity.com/paper/2412.11050/) (2024)
- [WebCiteS: Attributed Query-Focused Summarization on Chinese Web Search Results with Citations](https://mlsanity.com/paper/2403.01774/) (2024)
- [Talk to Right Specialists: Iterative Routing in Multi-agent Systems for Question Answering](https://mlsanity.com/paper/2501.07813/) (2025)
- [Principled Context Engineering for RAG: Statistical Guarantees via Conformal Prediction](https://mlsanity.com/paper/2511.17908/) (2025)
- [On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2502.14759/) (2025)
- [Large Language Model Sourcing: A Survey](https://mlsanity.com/paper/2510.10161/) (2025)
- [Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic Agents](https://mlsanity.com/paper/2506.01344/) (2025)
- [Plan Verification for LLM-Based Embodied Task Completion Agents](https://mlsanity.com/paper/2509.02761/) (2025)
- [An Iterative Utility Judgment Framework Inspired by Philosophical Relevance via LLMs](https://mlsanity.com/paper/2406.11290/) (2024)
- [ChartLens: Fine-grained Visual Attribution in Charts](https://mlsanity.com/paper/2505.19360/) (2025)
- [A Rigorous Evaluation of LLM Data Generation Strategies for Low-Resource Languages](https://mlsanity.com/paper/2506.12158/) (2025)
- [Zero-RAG: Towards Retrieval-Augmented Generation with Zero Redundant Knowledge](https://mlsanity.com/paper/2511.00505/) (2025)

## Abstract (arXiv)

Verifiable generation aims to let the large language model (LLM) generate text with supporting documents, which enables the user to flexibly verify the answer and makes the LLM's output more reliable. Retrieval plays a crucial role in verifiable generation. Specifically, the retrieved documents not only supplement knowledge to help the LLM generate correct answers, but also serve as supporting evidence for the user to verify the LLM's output. However, the widely used retrievers become the bottleneck of the entire pipeline and limit the overall performance. Their capabilities are usually inferior to LLMs since they often have much fewer parameters than the large language model and have not been demonstrated to scale well to the size of LLMs. If the retriever does not correctly find the supporting documents, the LLM can not generate the correct and verifiable answer, which overshadows the LLM's remarkable abilities. To address these limitations, we propose LLatrieval (Large Language Model Verified Retrieval), where the LLM updates the retrieval result until it verifies that the retrieved documents can sufficiently support answering the question. Thus, the LLM can iteratively provide feedback to retrieval and facilitate the retrieval result to fully support verifiable generation. Experiments show that LLatrieval significantly outperforms extensive baselines and achieves state-of-the-art results.