mlsanityData to 14 Oct 2026

Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data

arXiv 2010.05873 ↗First posted 12 Oct 2020cs.CLcited by 84

Not on the map. Why?

Abstract (arXiv)

Neural text generation (data- or text-to-text) demonstrates remarkable performance when training data is abundant which for many applications is not the case. To collect a large corpus of parallel data, heuristic rules are often used but they inevitably let noise into the data, such as phrases in the output which cannot be explained by the input. Consequently, models pick up on the noise and may hallucinate--generate fluent but unsupported text. Our contribution is a simple but powerful technique to treat such hallucinations as a controllable aspect of the generated text, without dismissing any input and without modifying the model architecture. On the WikiBio corpus (Lebret et al., 2016), a particularly noisy dataset, we demonstrate the efficacy of the technique both in an automatic and in a human evaluation.

Key facts. Cited by 84 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 6; 2023: 26; 2024: 22; 2025: 20; 2026: 1.

84 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  3. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  4. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  5. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  6. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 159
  7. 2021Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarizationcited by 129
  8. 2023Factuality Challenges in the Era of Large Language Modelscited by 126
  9. 2023Calibrated Language Models Must Hallucinatecited by 115
  10. 2025HalluLens: LLM Hallucination Benchmarkcited by 92
  11. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Modelscited by 90
  12. 2021Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selectioncited by 76
38 more
  1. 2021Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Featurescited by 74
  2. 2023Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedbackcited by 72
  3. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  4. 2024Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden Statescited by 52
  5. 2023Detecting and Mitigating Hallucinations in Multilingual Summarisationcited by 48
  6. 2023Exploring Human-Like Translation Strategy with Large Language Modelscited by 44
  7. 2024Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Modelscited by 44
  8. 2023How Ready are Pre-trained Abstractive Models and LLMs for Legal Case Judgement Summarization?cited by 43
  9. 2021Controlling Hallucinations at Word Level in Data-to-Text Generationcited by 41
  10. 2021Factual Consistency Evaluation for Text Summarization via Counterfactual Estimationcited by 37
  11. 2023Predictor-Rejector Multi-Class Abstention: Theoretical Analysis and Algorithmscited by 36
  12. 2024AI Hallucinations: A Misnomer Worth Clarifyingcited by 34
  13. 2023Learning to Reject with a Fixed Predictor: Application to Decontextualizationcited by 32
  14. 2023Theoretically Grounded Loss Functions and Algorithms for Score-Based Multi-Class Abstentioncited by 32
  15. 2024Measuring and Reducing LLM Hallucination without Gold-Standard Answerscited by 31
  16. 2024Improving Uncertainty Quantification in Large Language Models via Semantic Embeddingscited by 30
  17. 2022Learning to Revise References for Faithful Summarizationcited by 28
  18. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  19. 2024Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attackscited by 26
  20. 2021Sketch and Refine: Towards Faithful and Informative Table-to-Text Generationcited by 21
  21. 2022Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarizationcited by 20
  22. 2021SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasetscited by 18
  23. 2025Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Modelscited by 18
  24. 2023"Why is this misleading?": Detecting News Headline Hallucinations with Explanationscited by 17
  25. 2024An Audit on the Perspectives and Challenges of Hallucinations in NLPcited by 17
  26. 2024Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuningcited by 16
  27. 2023Quantifying and Attributing the Hallucination of Large Language Models via Association Analysiscited by 15
  28. 2021CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarizationcited by 11
  29. 2023Pointwise Mutual Information Based Metric and Decoding Strategy for Faithful Generation in Document Grounded Dialogscited by 11
  30. 2021Dissecting Generation Modes for Abstractive Summarization Models via Ablation and Attributioncited by 10
  31. 2022DeepGen: Diverse Search Ad Generation and Real-Time Customizationcited by 10
  32. 2022TaTa: A Multilingual Table-to-Text Dataset for African Languagescited by 10
  33. 2023Deep Learning Mental Health Dialogue Systemcited by 9
  34. 2025Shadows in the Attention: Contextual Perturbation and Representation Drift in the Dynamics of Hallucination in LLMscited by 9
  35. 2023What are the Desired Characteristics of Calibration Sets? Identifying Correlates on Long Form Scientific Summarizationcited by 8
  36. 2023Fidelity-Enriched Contrastive Search: Reconciling the Faithfulness-Diversity Trade-Off in Text Generationcited by 8
  37. 2025Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferencescited by 8
  38. 2025VeriCite: Towards Reliable Citations in Retrieval-Augmented Generation via Rigorous Verificationcited by 8

50 most cited of 84 All 84 (JSON)