# CBSiMT: Mitigating Hallucination in Simultaneous Machine Translation with Weighted Prefix-to-Prefix Training

arXiv: https://arxiv.org/abs/2311.03672 · HTML: https://mlsanity.com/paper/2311.03672/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 7 Nov 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [The Curious Case of Hallucinations in Neural Machine Translation](https://mlsanity.com/paper/2104.06683/) (2021)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [Detecting Hallucinated Content in Conditional Neural Sequence Generation](https://mlsanity.com/paper/2011.02593/) (2020)
- [On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation](https://mlsanity.com/paper/2005.03642/) (2020)
- [Learning to Translate in Real-time with Neural Machine Translation](https://mlsanity.com/paper/1610.00388/) (2016)
- [SimulEval: An Evaluation Toolkit for Simultaneous Translation](https://mlsanity.com/paper/2007.16193/) (2020)
- [Monotonic Multihead Attention](https://mlsanity.com/paper/1909.12406/) (2019)
- [Calibration of Encoder Decoder Models for Neural Machine Translation](https://mlsanity.com/paper/1903.00802/) (2019)
- [Domain Robustness in Neural Machine Translation](https://mlsanity.com/paper/1911.03109/) (2019)
- [On the Inference Calibration of Neural Machine Translation](https://mlsanity.com/paper/2005.00963/) (2020)
- [Efficient Wait-k Models for Simultaneous Machine Translation](https://mlsanity.com/paper/2005.08595/) (2020)
- [Simultaneous Translation Policies: From Fixed to Adaptive](https://mlsanity.com/paper/2004.13169/) (2020)
- [Information-Transport-based Policy for Simultaneous Translation](https://mlsanity.com/paper/2210.12357/) (2022)
- [Improving Back-Translation with Uncertainty-based Confidence Estimation](https://mlsanity.com/paper/1909.00157/) (2019)
- [Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy](https://mlsanity.com/paper/2109.05238/) (2021)
- [Future-Guided Incremental Transformer for Simultaneous Translation](https://mlsanity.com/paper/2012.12465/) (2020)
- [Self-Paced Learning for Neural Machine Translation](https://mlsanity.com/paper/2010.04505/) (2020)
- [Modeling Dual Read/Write Paths for Simultaneous Machine Translation](https://mlsanity.com/paper/2203.09163/) (2022)
- [Improving Simultaneous Translation by Incorporating Pseudo-References with Fewer Reorderings](https://mlsanity.com/paper/2010.11247/) (2020)
- [Gaussian Multi-head Attention for Simultaneous Machine Translation](https://mlsanity.com/paper/2203.09072/) (2022)
- [Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework](https://mlsanity.com/paper/2203.09053/) (2022)
- [Confidence-Aware Scheduled Sampling for Neural Machine Translation](https://mlsanity.com/paper/2107.10427/) (2021)
- [Anticipation-Free Training for Simultaneous Machine Translation](https://mlsanity.com/paper/2201.12868/) (2022)

## Built on by (most cited first, 1 of 1)

- [On the Hallucination in Simultaneous Machine Translation](https://mlsanity.com/paper/2406.07239/) (2024)

## Abstract (arXiv)

Simultaneous machine translation (SiMT) is a challenging task that requires starting translation before the full source sentence is available. Prefix-to-prefix framework is often applied to SiMT, which learns to predict target tokens using only a partial source prefix. However, due to the word order difference between languages, misaligned prefix pairs would make SiMT models suffer from serious hallucination problems, i.e. target outputs that are unfaithful to source inputs. Such problems can not only produce target tokens that are not supported by the source prefix, but also hinder generating the correct translation by receiving more source words. In this work, we propose a Confidence-Based Simultaneous Machine Translation (CBSiMT) framework, which uses model confidence to perceive hallucination tokens and mitigates their negative impact with weighted prefix-to-prefix training. Specifically, token-level and sentence-level weights are calculated based on model confidence and acted on the loss function. We explicitly quantify the faithfulness of the generated target tokens using the token-level weight, and employ the sentence-level weight to alleviate the disturbance of sentence pairs with serious word order differences on the model. Experimental results on MuST-C English-to-Chinese and WMT15 German-to-English SiMT tasks demonstrate that our method can consistently improve translation quality at most latency regimes, with up to 2 BLEU scores improvement at low latency.