mlsanityData to 7 Oct 2026

Mutual Information and Diverse Decoding Improve Neural Machine Translation

arXiv 1601.00372 ↗First posted 4 Jan 2016cs.CL cs.AIcited by 35

Not on the map. Why?

Abstract (arXiv)

Sequence-to-sequence neural translation models learn semantic and syntactic relations between sentence pairs by optimizing the likelihood of the target given the source, i.e., p(y|x), an objective that ignores other potentially useful sources of information. We introduce an alternative objective function for neural MT that maximizes the mutual information between the source and target sentences, modeling the bi-directional dependency of sources and targets. We implement the model with a simple re-ranking method, and also introduce a decoding algorithm that increases diversity in the N-best list produced by the first pass. Applied to the WMT German/English and French/English tasks, the proposed models offers a consistent performance boost on both standard LSTM and attention-based neural MT architectures.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Jan 2016.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 10; 2022: 3; 2023: 6; 2024: 10; 2025: 6.

35 papers.

Sort:

By area:

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2022On Decoding Strategies for Neural Text Generatorscited by 89
  3. 2025Jointly Reinforcing Diversity and Quality in Language Model Generationscited by 79
  4. 2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaborationcited by 70
  5. 2021Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translationcited by 63
  6. 2023Revisiting the Role of Language Priors in Vision-Language Modelscited by 50
  7. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  8. 2021Fast Nearest Neighbor Machine Translationcited by 37
  9. 2023The Consensus Game: Language Model Generation via Equilibrium Searchcited by 35
  10. 2023Benchmarking and Improving Text-to-SQL Generation under Ambiguitycited by 31
  11. 2022Few-shot Subgoal Planning with Language Modelscited by 22
  12. 2023Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Surveycited by 18
23 more
  1. 2024Metasql: A Generate-then-Rank Framework for Natural Language to SQL Translationcited by 17
  2. 2024Improving Group Robustness on Spurious Correlation Requires Preciser Group Inferencecited by 15
  3. 2025ETS: Efficient Tree Search for Inference-Time Scalingcited by 15
  4. 2021ConRPG: Paraphrase Generation using Contexts as Regularizercited by 13
  5. 2024Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decodingcited by 11
  6. 2021Faster Nearest Neighbor Machine Translationcited by 8
  7. 2025Incentivizing Truthful Language Models via Peer Elicitation Gamescited by 8
  8. 2025Pragmatic Reasoning improves LLM Code Generationcited by 6
  9. 2021Cooperative Self-training of Machine Reading Comprehensioncited by 5
  10. 2021REAMsharp: An Enhancement Approach to Reference-based Evaluation Metrics for Open-domain Dialog Generationcited by 4
  11. 2021RewardsOfSum: Exploring Reinforcement Learning Rewards for Summarisationcited by 4
  12. 2025Multi-Amateur Contrastive Decoding for Text Generationcited by 3
  13. 2024Generating Diverse Translation with Perturbed kNN-MTcited by 2
  14. 2024Truth or Deceit? A Bayesian Decoding Game Enhances Consistency and Reliabilitycited by 2
  15. 2024Self-Explained Keywords Empower Large Language Models for Code Generationcited by 2
  16. 2025Geometry of Knowledge Allows Extending Diversity Boundaries of Large Language Modelscited by 2
  17. 2021There Once Was a Really Bad Poet, It Was Automated but You Didn't Know Itcited by 1
  18. 2021Goal-driven text descriptions for imagescited by 1
  19. 2023Bidirectional Transformer Reranker for Grammatical Error Correctioncited by 1
  20. 2024Multi-party Response Generation with Relation Disentanglementcited by 0
  21. 2024Which Information Matters? Dissecting Human-written Multi-document Summaries with Partial Information Decompositioncited by 0
  22. 2024Plug, Play, and Fuse: Zero-Shot Joint Decoding via Word-Level Re-ranking Across Diverse Vocabulariescited by 0
  23. 2024Cross-Lingual Text-Rich Visual Comprehension: An Information Theory Perspectivecited by 0