Enhanced Simultaneous Machine Translation with Word-level Policies
Not on the map. Why?
Abstract (arXiv)
Recent years have seen remarkable advances in the field of Simultaneous Machine Translation (SiMT) due to the introduction of innovative policies that dictate whether to READ or WRITE at each step of the translation process. However, a common assumption in many existing studies is that operations are carried out at the subword level, even though the standard unit for input and output in most practical scenarios is typically at the word level. This paper demonstrates that policies devised and validated at the subword level are surpassed by those operating at the word level, which process multiple subwords to form a complete word in a single step. Additionally, we suggest a method to boost SiMT models using language models (LMs), wherein the proposed word-level policy plays a vital role in addressing the subword disparity between LMs and SiMT models. Code is available at https://github.com/xl8-ai/WordSiMT.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 37 of its 49 references that are in the map. First posted on arXiv 25 Oct 2023.
Builds on
37 of 49 references.
References · 37
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
- 2019Cross-lingual Language Model Pretrainingcited by 1,088
- 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
- 2021Few-shot Learning with Multilingual Language Modelscited by 278
- 2019Monotonic Infinite Lookback Attention for Simultaneous Machine Translationcited by 103
- 2016Learning to Translate in Real-time with Neural Machine Translationcited by 97
25 more
- 2019Monotonic Multihead Attentioncited by 91
- 2016Can neural machine translation do simultaneous translation?cited by 85
- 2017Online and Linear-Time Attention by Enforcing Monotonic Alignmentscited by 85
- 2020Efficient Wait-k Models for Simultaneous Machine Translationcited by 59
- 2018Incremental Decoding and Training Methods for Simultaneous Translation in Neural Machine Translationcited by 51
- 2019Towards Making the Most of BERT in Neural Machine Translationcited by 43
- 2020Simultaneous Translation Policies: From Fixed to Adaptivecited by 42
- 2022Information-Transport-based Policy for Simultaneous Translationcited by 40
- 2021BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translationcited by 35
- 2021Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policycited by 32
- 2019Acquiring Knowledge from Pre-trained Model to Neural Machine Translationcited by 30
- 2020Future-Guided Incremental Transformer for Simultaneous Translationcited by 30
- 2020Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translationcited by 28
- 2019Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabulariescited by 27
- 2023Hidden Markov Transformer for Simultaneous Machine Translationcited by 24
- 2022Modeling Dual Read/Write Paths for Simultaneous Machine Translationcited by 21
- 2020Simultaneous Machine Translation with Visual Contextcited by 17
- 2022Gaussian Multi-head Attention for Simultaneous Machine Translationcited by 17
- 2021Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLUcited by 16
- 2022Wait-info Policy: Balancing Source and Target at Information Level for Simultaneous Machine Translationcited by 16
- 2021On the Copying Behaviors of Pre-Training for Neural Machine Translationcited by 15
- 2023Learning Optimal Policy for Simultaneous Machine Translation via Binary Searchcited by 15
- 2021Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generationcited by 14
- 2022Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine Translationcited by 14
- 2022From Simultaneous to Streaming Machine Translation by Leveraging Streaming Historycited by 9
Built on by
Citing papers per year: 2024: 2.
2 papers.