mlsanityData to 7 Oct 2026

Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding

arXiv 2309.07098 ↗First posted 13 Sep 2023cs.CLcited by 42

Not on the map. Why?

Abstract (arXiv)

Hallucinations and off-target translation remain unsolved problems in MT, especially for low-resource languages and massively multilingual models. In this paper, we introduce two related methods to mitigate these failure cases with a modified decoding objective, without either requiring retraining or external models. In source-contrastive decoding, we search for a translation that is probable given the correct input, but improbable given a random input segment. In language-contrastive decoding, we search for a translation that is probable, but improbable given the wrong language indicator token. Experiments on the massively multilingual models M2M-100 (418M) and SMaLL-100 show that these methods suppress hallucinations and off-target translations, reducing the number of translations with segment-level chrF2 below 10 by 67-83% on average, and the number of translations with oscillatory hallucinations by 75-92% on average, across 57 tested translation directions. In a proof of concept on out-of-English translation, we also show that we can suppress off-target translations with large language models. We release our source code at https://github.com/ZurichNLP/ContraDecode.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 40 of its 49 references that are in the map. First posted on arXiv 13 Sep 2023.

Builds on

40 of 49 references.

References · 40

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  5. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  8. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  9. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  10. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  11. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  12. 2017Six Challenges for Neural Machine Translationcited by 334
28 more
  1. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  2. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
  3. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  4. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  5. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 144
  6. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 137
  7. 2018Analyzing Uncertainty in Neural Machine Translationcited by 126
  8. 2023Hallucinations in Large Multilingual Translation Modelscited by 126
  9. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  10. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  11. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  12. 2018On the Impact of Various Types of Noise on Neural Machine Translationcited by 76
  13. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  14. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 60
  15. 2019The Missing Ingredient in Zero-Shot Neural Machine Translationcited by 59
  16. 2022Contrastive Search Is What You Need For Neural Text Generationcited by 58
  17. 2023Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOMcited by 58
  18. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  19. 2023An Open Dataset and Model for Language Identificationcited by 49
  20. 2016Domain Control for Neural Machine Translationcited by 36
  21. 2023BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Trainingcited by 24
  22. 2022SALTED: A Framework for SAlient Long-Tail Translation Error Detectioncited by 23
  23. 2018Multilingual NMT with a language-independent attention bridgecited by 20
  24. 2022SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languagescited by 16
  25. 2020Subword Segmentation and a Single Bridge Language Affect Zero-Shot Neural Machine Translationcited by 13
  26. 2022As Little as Possible, as Much as Necessary: Detecting Over- and Undertranslations with Contrastive Conditioningcited by 11
  27. 2022What Do Compressed Multilingual Machine Translation Models Forget?cited by 9
  28. 2023Improving Language Model Integration for Neural Machine Translationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 21; 2025: 12; 2026: 6.

42 papers.

Sort:

By area:

  1. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decodingcited by 198
  2. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 124
  3. 2024Tuning Language Models by Proxycited by 88
  4. 2024Understanding and Mitigating Language Confusion in LLMscited by 75
  5. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  6. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  7. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  8. 2024ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decodingcited by 32
  9. 2024CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Followingcited by 32
  10. 2023Closing the Curious Case of Neural Text Degenerationcited by 30
  11. 2025When One LLM Drools, Multi-LLM Collaboration Rulescited by 25
  12. 2024Mitigating Hallucinations in Large Vision-Language Models (LVLMs) via Language-Contrastive Decoding (LCD)cited by 23
30 more
  1. 2024Selective Prompt Anchoring for Code Generationcited by 16
  2. 2025G2: Guided Generation for Enhanced Output Diversity in LLMscited by 14
  3. 2024Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentationcited by 13
  4. 2024Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decodingcited by 12
  5. 2025Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generationcited by 10
  6. 2024Enhanced Hallucination Detection in Neural Machine Translation through Simple Detector Aggregationcited by 9
  7. 2024Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Promptscited by 9
  8. 2025Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Modelscited by 8
  9. 2024Large Language Models are Easily Confused: A Quantitative Metric, Security Implications and Typological Analysiscited by 7
  10. 2025Poly-FEVER: A Multilingual Fact Verification Benchmark for Hallucination Detection in Large Language Modelscited by 7
  11. 2024mathbbUSCD: Improving Code Generation of LLMs by Uncertainty-Aware Selective Contrastive Decodingcited by 6
  12. 2025Mitigating Hallucinated Translations in Large Language Models with Hallucination-focused Preference Optimizationcited by 6
  13. 2023Investigating Multi-Pivot Ensembling with Massively Multilingual Machine Translation Modelscited by 5
  14. 2024Lens: Rethinking Multilingual Enhancement for Large Language Modelscited by 5
  15. 2025Enhancing Input-Label Mapping in In-Context Learning with Contrastive Decodingcited by 4
  16. 2024Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuningcited by 3
  17. 2024CogSteer: Cognition-Inspired Selective Layer Intervention for Efficiently Steering Large Language Modelscited by 3
  18. 2026Whisper-CD: Accurate Long-Form Speech Recognition using Multi-Negative Contrastive Decodingcited by 3
  19. 2023Anti-LM Decoding for Zero-shot In-context Machine Translationcited by 2
  20. 2024Language-Informed Beam Search Decoding for Multilingual Machine Translationcited by 2
  21. 2025Synthetic Fluency: Hallucinations, Confabulations, and the Creation of Irish Words in LLM-Generated Translationscited by 2
  22. 2025HalluCounter: Reference-free LLM Hallucination Detection in the Wild!cited by 1
  23. 2025Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Modelscited by 1
  24. 2025The Effect of Language Diversity When Fine-Tuning Large Language Models for Translationcited by 1
  25. 2026Seeing Right but Saying Wrong: Inter- and Intra-Layer Refinement in MLLMs without Trainingcited by 1
  26. 2026Is MT Ready for the Next Crisis or Pandemic?cited by 1
  27. 2024How Far Can 100 Samples Go? Unlocking Overall Zero-Shot Multilingual Translation via Tiny Multi-Parallel Datacited by 0
  28. 2026Self-Correction Inside the Model: Leveraging Layer Attention to Mitigate Hallucinations in Large Vision Language Modelscited by 0
  29. 2026HTDC: Hesitation-Triggered Differential Calibration for Mitigating Hallucination in Large Vision-Language Modelscited by 0
  30. 2026Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusioncited by 0