mlsanityData to 7 Oct 2026

Alleviating Hallucinations of Large Language Models through Induced Hallucinations

arXiv 2312.15710 ↗First posted 25 Dec 2023cs.CL cs.AIcited by 78

Not on the map. Why?

Abstract (arXiv)

Despite their impressive capabilities, large language models (LLMs) have been observed to generate responses that include inaccurate or fabricated information, a phenomenon commonly known as ``hallucination''. In this work, we propose a simple textitInduce-then-Contrast Decoding (ICD) strategy to alleviate hallucinations. We first construct a factually weak LLM by inducing hallucinations from the original LLMs. Then, we penalize these induced hallucinations during decoding to enhance the factuality of the generated content. Concretely, we determine the final next-token predictions by amplifying the predictions from the original model and downplaying the induced untruthful predictions via contrastive decoding. Experimental results on both discrimination-based and generation-based hallucination evaluation benchmarks, such as TruthfulQA and textscFActScore, demonstrate that our proposed ICD methods can effectively enhance the factuality of LLMs across various model sizes and families. For example, when equipped with ICD, Llama2-7B-Chat and Mistral-7B-Instruct achieve performance comparable to ChatGPT and GPT4 on TruthfulQA, respectively.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Builds on 47 of its 65 references that are in the map. First posted on arXiv 25 Dec 2023.

Builds on

47 of 65 references.

References · 47

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2023Let's Verify Step by Stepcited by 3,598
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  12. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
35 more
  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  5. 2023LIMA: Less Is More for Alignmentcited by 1,080
  6. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  7. 2022Red Teaming Language Models with Language Modelscited by 983
  8. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  9. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  10. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  11. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  12. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  13. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  14. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  15. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  16. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  17. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  18. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  19. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  20. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  21. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  22. 2023Fine-tuning Language Models for Factualitycited by 219
  23. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  24. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  25. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 168
  26. 2022A Survey on Retrieval-Augmented Text Generationcited by 157
  27. 2023Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluationcited by 97
  28. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 89
  29. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  30. 2023Do Large Language Models Know about Facts?cited by 49
  31. 2023DelucionQA: Detecting Hallucinations in Domain-specific Question Answeringcited by 43
  32. 2023Multi-Task Instruction Tuning of LLaMa for Specific Scenarios: A Preliminary Study on Writing Assistancecited by 33
  33. 2023Surfacing Biases in Large Language Models using Contrastive Input Decodingcited by 11
  34. 2023HistAlign: Improving Context Dependency in Language Generation by Aligning with Historycited by 6
  35. 2023Hallucination Reduction in Long Input Text Summarizationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 39; 2025: 25; 2026: 14.

78 papers.

Sort:

By area:

  1. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
  2. 2024TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Spacecited by 90
  3. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 85
  4. 2024Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capabilitycited by 80
  5. 2024Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluationcited by 79
  6. 2024Unified Hallucination Detection for Multimodal Large Language Modelscited by 75
  7. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 54
  8. 2024When to Trust LLMs: Aligning Confidence with Response Qualitycited by 42
  9. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  10. 2024Quantifying the Gain in Weak-to-Strong Generalizationcited by 40
  11. 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Modelscited by 38
  12. 2024Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Factscited by 36
38 more
  1. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  2. 2024Mitigating Large Language Model Hallucination with Faithful Finetuningcited by 24
  3. 2024Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Modelscited by 24
  4. 2024DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinationscited by 23
  5. 2024A Survey on the Honesty of Large Language Modelscited by 21
  6. 2025Cross-Modal Attention Calibration for LVLM Hallucination Mitigationcited by 21
  7. 2024OLAPH: Improving Factuality in Biomedical Long-form Question Answeringcited by 20
  8. 2024Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answeringcited by 20
  9. 2024Improving Factuality in Large Language Models via Decoding-Time Hallucinatory and Truthful Comparatorscited by 19
  10. 2024Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reductioncited by 18
  11. 2024Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulnesscited by 17
  12. 2024SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Modelscited by 16
  13. 2025Advancing LLM Safe Alignment with Safety Representation Rankingcited by 15
  14. 2024Pensieve: Retrospect-then-Compare Mitigates Visual Hallucinationcited by 14
  15. 2024Adaptive Token Biaser: Knowledge Editing via Biasing Key Entitiescited by 14
  16. 2024Multi-group Uncertainty Quantification for Long-form Text Generationcited by 13
  17. 2024Adversarial Representation Engineering: A General Model Editing Framework for Large Language Modelscited by 12
  18. 2024Explaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LMcited by 11
  19. 2024FACTOID: FACtual enTailment fOr hallucInation Detectioncited by 10
  20. 2024Collaborative decoding of critical tokens for boosting factuality of large language modelscited by 9
  21. 2024Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Promptscited by 9
  22. 2024NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Modelscited by 9
  23. 2025Unified attacks to large language model watermarks: spoofing and scrubbing in unauthorized knowledge distillationcited by 9
  24. 2025SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinationscited by 9
  25. 2024MAO: A Framework for Process Model Generation with Multi-Agent Orchestrationcited by 8
  26. 2024Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decodingcited by 8
  27. 2026Hallucination Begins Where Saliency Dropscited by 8
  28. 2024FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Modelscited by 7
  29. 2025Improve Decoding Factuality by Token-wise Cross Layer Entropy of Large Language Modelscited by 7
  30. 2025SAFE: A Sparse Autoencoder-Based Framework for Robust Query Enrichment and Hallucination Mitigation in LLMscited by 7
  31. 2025HICD: Hallucination-Inducing via Attention Dispersion for Contrastive Decoding to Mitigate Hallucinations in Large Language Modelscited by 7
  32. 2025Evaluating Evaluation Metrics -- The Mirage of Hallucination Detectioncited by 6
  33. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 6
  34. 2026Why Fine-Tuning Encourages Hallucinations and How to Fix Itcited by 6
  35. 2024Enhancing Hallucination Detection through Perturbation-Based Synthetic Data Generation in System Responsescited by 5
  36. 2025A Survey on Proactive Defense Strategies Against Misinformation in Large Language Modelscited by 5
  37. 2024Gaps or Hallucinations? Gazing into Machine-Generated Legal Analysis for Fine-grained Text Evaluationscited by 4
  38. 2025Exploring the Generalizability of Factual Hallucination Mitigation via Enhancing Precise Knowledge Utilizationcited by 4

50 most cited of 78 All 78 (JSON)