mlsanityData to 7 Oct 2026

Teaching language models to support answers with verified quotes

arXiv 2203.11147 ↗First posted 21 Mar 2022cs.CL cs.LGcited by 290

Not on the map. Why?

Cited by 55 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent large language models often answer factual questions correctly. But users can't trust any given claim a model makes without fact-checking, because language models can hallucinate convincing nonsense. In this work we use reinforcement learning from human preferences (RLHP) to train "open-book" QA models that generate answers whilst also citing specific evidence for their claims, which aids in the appraisal of correctness. Supporting evidence is drawn from multiple documents found via a search engine, or from a single user-provided document. Our 280 billion parameter model, GopherCite, is able to produce answers with high quality supporting evidence and abstain from answering when unsure. We measure the performance of GopherCite by conducting human evaluation of answers to questions in a subset of the NaturalQuestions and ELI5 datasets. The model's response is found to be high-quality 80% of the time on this Natural Questions subset, and 67% of the time on the ELI5 subset. Abstaining from the third of questions for which it is most unsure improves performance to 90% and 80% respectively, approaching human baselines. However, analysis on the adversarial TruthfulQA dataset shows why citation is only one part of an overall strategy for safety and trustworthiness: not all claims supported by evidence are true.

Key facts. Cited by 290 papers in the map (data through 7 Oct 2026). Builds on 50 of its 73 references that are in the map. First posted on arXiv 21 Mar 2022.

Builds on

50 of 73 references.

References · 50

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2020Learning to summarize from human feedbackcited by 2,562
  10. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  12. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
38 more
  1. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  2. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  3. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  4. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  5. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  6. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  7. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  8. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  9. 2022Red Teaming Language Models with Language Modelscited by 983
  10. 2021Ethical and social risks of harm from Language Modelscited by 957
  11. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  12. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  13. 2017Selective Classification for Deep Neural Networkscited by 799
  14. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  15. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  16. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  17. 2019ELI5: Long Form Question Answeringcited by 567
  18. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  19. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  20. 2018AI safety via debatecited by 384
  21. 2015Deep Learning with Limited Numerical Precisioncited by 368
  22. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
  23. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  24. 2020Selective Question Answering under Domain Shiftcited by 228
  25. 2021Internet-Augmented Dialogue Generationcited by 217
  26. 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 204
  27. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  28. 2020WT5?! Training Text-to-Text Models to Explain their Predictionscited by 148
  29. 2018Supervising strong learners by amplifying weak expertscited by 139
  30. 2020Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasetscited by 137
  31. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  32. 2020Explainable Deep Learning: A Field Guide for the Uninitiatedcited by 127
  33. 2016Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-controlcited by 100
  34. 2018Kickstarting Deep Reinforcement Learningcited by 75
  35. 2020QED: A Framework and Dataset for Explanations in Question Answeringcited by 55
  36. 2021UnitedQA: A Hybrid Approach for Open Domain Question Answeringcited by 42
  37. 2020Explaining Question Answering Models through Text Generationcited by 33
  38. 2019Finding Generalizable Evidence by Learning to Convince Q&A Modelscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 17; 2023: 97; 2024: 82; 2025: 47; 2026: 47.

290 papers.

Sort:

By area:

  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  3. 2023A Survey of Large Language Modelscited by 2,537
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  7. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  8. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  9. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  10. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  11. 2022A Generalist Agentcited by 802
  12. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
38 more
  1. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  2. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  3. 2023A Comprehensive Overview of Large Language Modelscited by 639
  4. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  5. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  6. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  7. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  8. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  9. 2023Augmented Language Models: a Surveycited by 398
  10. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  11. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  12. 2023Language Models can Solve Computer Taskscited by 371
  13. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  14. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  15. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  16. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  17. 2023Tool Learning with Foundation Modelscited by 302
  18. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  19. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  20. 2023Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Mergingcited by 271
  21. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  22. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
  23. 2023Pretraining Language Models with Human Preferencescited by 245
  24. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 244
  25. 2023Nash Learning from Human Feedbackcited by 220
  26. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  27. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 203
  28. 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
  29. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  30. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  31. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  32. 2023PaperQA: Retrieval-Augmented Generative Agent for Scientific Researchcited by 143
  33. 2023Vision-Language Models as Success Detectorscited by 138
  34. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 136
  35. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
  36. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  37. 2024Personalized Language Modeling from Personalized Human Feedbackcited by 124
  38. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118

50 most cited of 290 All 290 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026An Analysis of Training-Free Self-Reported Confidence in Language Models
  2. 2024Language Models Learn to Mislead Humans via RLHF
  3. 2025Teaching Large Language Models to Maintain Contextual Faithfulness via Synthetic Tasks and Reinforcement Learning
  4. 2025Trustworthy Reasoning: Evaluating and Enhancing Factual Accuracy in LLM Intermediate Thought Processes
  5. 2026Fact-Checking with Large Language Models via Probabilistic Certainty and Consistency
  6. 2025Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models
  7. 2023Fine-tuning Language Models for Factuality
  8. 2025Learning to Reason for Factuality