mlsanityData to 7 Oct 2026

PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptions

arXiv 2305.14908 ↗First posted 24 May 2023cs.CLcited by 53

Not on the map. Why?

Abstract (arXiv)

The remarkable capabilities of large language models have been accompanied by a persistent drawback: the generation of false and unsubstantiated claims commonly known as "hallucinations". To combat this issue, recent research has introduced approaches that involve editing and attributing the outputs of language models, particularly through prompt-based editing. However, the inference cost and speed of using large language models for editing currently bottleneck prompt-based methods. These bottlenecks motivate the training of compact editors, which is challenging due to the scarcity of training data for this purpose. To overcome these challenges, we exploit the power of large language models to introduce corruptions (i.e., noise) into text and subsequently fine-tune compact editors to denoise the corruptions by incorporating relevant evidence. Our methodology is entirely unsupervised and provides us with faux hallucinations for training in any domain. Our Petite Unsupervised Research and Revision model, PURR, not only improves attribution over existing editing methods based on fine-tuning and prompting, but also achieves faster execution times by orders of magnitude.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 38 of its 43 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

38 of 43 references.

References · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2020Learning to summarize from human feedbackcited by 2,562
  7. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  8. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  9. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  10. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  11. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  12. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
26 more
  1. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  2. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  3. 2021MetaICL: Learning to Learn In Contextcited by 450
  4. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  5. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  6. 2021Knowledge distillation: A good teacher is patient and consistentcited by 255
  7. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  8. 2023Pretraining Language Models with Human Preferencescited by 245
  9. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  10. 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
  11. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  12. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  13. 2020Open-Domain Question Answering Goes Conversational via Question Rewritingcited by 147
  14. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  15. 2020Factual Error Correction for Abstractive Summarization Modelscited by 108
  16. 2022PEER: A Collaborative Language Modelcited by 86
  17. 2022Language Models in the Loop: Incorporating Prompting into Weak Supervisioncited by 51
  18. 2020Evidence-based Factual Error Correctioncited by 44
  19. 2021FRUIT: Faithfully Reflecting Updated Information in Textcited by 39
  20. 2022Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infillingcited by 38
  21. 2021Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistantscited by 31
  22. 2022Co-training Improves Prompt-based Learning for Large Language Modelscited by 24
  23. 2022Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learnerscited by 22
  24. 2022Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Modelcited by 20
  25. 2020Fact-based Text Editingcited by 14
  26. 2021Simple and Efficient ways to Improve REALMcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 14; 2024: 26; 2025: 8; 2026: 5.

53 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  3. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  4. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  5. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 146
  6. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 130
  7. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 106
  8. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 98
  9. 2024ContextCite: Attributing Model Generation to Contextcited by 93
  10. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  11. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 87
  12. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
38 more
  1. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  2. 2023Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofittingcited by 70
  3. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  4. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  5. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  6. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44
  7. 2023Exploring the Numerical Reasoning Capabilities of Language Models: A Comprehensive Analysis on Tabular Datacited by 33
  8. 2024Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMscited by 30
  9. 2024A Survey of Conversational Searchcited by 22
  10. 2024Atomic Fact Decomposition Helps Attributed Question Answeringcited by 21
  11. 2024Correctness is not Faithfulness in RAG Attributionscited by 18
  12. 2025Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Modelscited by 18
  13. 2024Hallucination Detection in Foundation Models for Decision-Making: A Flexible Definition and Review of the State of the Artcited by 17
  14. 2024Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuningcited by 16
  15. 2023Building Trustworthy NeuroSymbolic AI Systems: Consistency, Reliability, Explainability, and Safetycited by 14
  16. 2024Improving Faithfulness of Large Language Models in Summarization via Sliding Generation and Self-Consistencycited by 14
  17. 2024SYNFAC-EDIT: Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarizationcited by 12
  18. 2025Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generationcited by 12
  19. 2024Mitigating Memorization In Language Modelscited by 11
  20. 2024The Extractive-Abstractive Spectrum: Uncovering Verifiability Trade-offs in LLM Generationscited by 8
  21. 2025Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metricscited by 8
  22. 2024Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Modelscited by 7
  23. 2025Attribution, Citation, and Quotation: A Survey of Evidence-based Text Generation with Large Language Modelscited by 7
  24. 2023Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarizationcited by 5
  25. 2024Medical Dialogue: A Survey of Categories, Methods, Evaluation and Challengescited by 5
  26. 2024Analysis of Plan-based Retrieval for Grounded Text Generationcited by 5
  27. 2025ZINA: Multimodal Fine-grained Hallucination Detection and Editingcited by 5
  28. 2026When Memory Updates but Behavior Does Not: Repairing Implicit Stale Dependencies in Personalized Agent Responsescited by 5
  29. 2023KGQuiz: Evaluating the Generalization of Encoded Knowledge in Large Language Modelscited by 4
  30. 2023Defining a New NLP Playgroundcited by 4
  31. 2025GenerationPrograms: Fine-grained Attribution with Executable Programscited by 4
  32. 2024Embedded Named Entity Recognition using Probing Classifierscited by 3
  33. 2024Fennec: Fine-grained Language Model Evaluation and Correction Extended through Branching and Bridgingcited by 3
  34. 2026Who Checks the Citations? Benchmarking Legal Hallucination Detectioncited by 2
  35. 2024Rate, Explain and Cite (REC): Enhanced Explanation and Attribution in Automatic Evaluation by Large Language Modelscited by 1
  36. 2025MALM: A Multi-Information Adapter for Large Language Models to Mitigate Hallucinationcited by 1
  37. 2024QUILL: Quotation Generation Enhancement of Large Language Modelscited by 0
  38. 2025Hallucination Detection and Mitigation with Diffusion in Multi-Variate Time-Series Foundation Modelscited by 0

50 most cited of 53 All 53 (JSON)