mlsanityData to 14 Oct 2026

Improving Simultaneous Machine Translation with Monolingual Data

arXiv 2212.01188 ↗First posted 2 Dec 2022cs.CLcited by 13

On the map: Knowledge distillation → Knowledge distillation for LLMs

Abstract (arXiv)

Simultaneous machine translation (SiMT) is usually done via sequence-level knowledge distillation (Seq-KD) from a full-sentence neural machine translation (NMT) model. However, there is still a significant performance gap between NMT and SiMT. In this work, we propose to leverage monolingual data to improve SiMT, which trains a SiMT student on the combination of bilingual data and external monolingual data distilled by Seq-KD. Preliminary experiments on En-Zh and En-Ja news domain corpora demonstrate that monolingual data can significantly improve translation quality (e.g., +3.15 BLEU on En-Zh). Inspired by the behavior of human simultaneous interpreters, we propose a novel monolingual sampling strategy for SiMT, considering both chunk length and monotonicity. Experimental results show that our sampling strategy consistently outperforms the random sampling strategy (and other conventional typical NMT monolingual sampling strategies) by avoiding the key problem of SiMT -- hallucination, and has better scalability. We achieve +0.72 BLEU improvements on average against random sampling on En-Zh and En-Ja. Data and codes can be found at https://github.com/hexuandeng/Mono4SiMT.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 32 of its 57 references that are in the map. First posted on arXiv 2 Dec 2022.

Builds on

32 of 57 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation2 Jun

Other references · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  6. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  7. 2016Sequence-Level Knowledge Distillationcited by 801
  8. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  9. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  10. 2017Six Challenges for Neural Machine Translationcited by 335
  11. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 223
  12. 2019Competence-based Curriculum Learning for Neural Machine Translationcited by 202
19 more
  1. 2019Understanding Knowledge Distillation in Non-autoregressive Machine Translationcited by 122
  2. 2019Monotonic Infinite Lookback Attention for Simultaneous Machine Translationcited by 104
  3. 2016Learning to Translate in Real-time with Neural Machine Translationcited by 97
  4. 2020SimulEval: An Evaluation Toolkit for Simultaneous Translationcited by 94
  5. 2020Norm-Based Curriculum Learning for Neural Machine Translationcited by 60
  6. 2020Understanding and Improving Lexical Choice in Non-Autoregressive Translationcited by 56
  7. 2020Simultaneous Translation Policies: From Fixed to Adaptivecited by 43
  8. 2022Information-Transport-based Policy for Simultaneous Translationcited by 40
  9. 2021Prevent the Language Model from being Overconfident in Neural Machine Translationcited by 34
  10. 2020Future-Guided Incremental Transformer for Simultaneous Translationcited by 30
  11. 2019JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpuscited by 21
  12. 2020Improving Simultaneous Translation by Incorporating Pseudo-References with Fewer Reorderingscited by 20
  13. 2018Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translationcited by 19
  14. 2021Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translationcited by 17
  15. 2021The USTC-NELSLIP Systems for Simultaneous Speech Translation Task at IWSLT 2021cited by 13
  16. 2021The USYD-JD Speech Translation System for IWSLT 2021cited by 11
  17. 2020Improving Non-autoregressive Neural Machine Translation with Monolingual Datacited by 10
  18. 2021Monotonic Simultaneous Translation with Chunk-wise Reordering and Refinementcited by 4
  19. 2021The Volctrans Neural Speech Translation System for IWSLT 2021cited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 3; 2026: 2.

13 papers.

Sort:

By area:

  1. 2023Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?cited by 21
  2. 2024Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillationcited by 12
  3. 2025APT: Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Trainingcited by 12
  4. 2024NewTerm: Benchmarking Real-Time New Terms for Large Language Models with Annual Updatescited by 8
  5. 2023Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inferencecited by 6
  6. 2024DRPruning: Efficient Large Language Model Pruning through Distributionally Robust Optimizationcited by 5
  7. 2025LLMs Can Achieve High-quality Simultaneous Machine Translation as Efficiently as Offlinecited by 5
  8. 2025AQuilt: Weaving Logic and Self-Inspection into Low-Cost, High-Relevance Data Synthesis for Specialist LLMscited by 4
  9. 2026Stop Rewarding Hallucinated Steps: Faithfulness-Aware Step-Level Reinforcement Learning for Small Reasoning Modelscited by 4
  10. 2024Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Paircited by 3
  11. 2023Holistic Exploration on Universal Decompositional Semantic Parsing: Architecture, Data Augmentation, and LLM Paradigmcited by 2
  12. 2026KD4MT: A Survey of Knowledge Distillation for Machine Translationcited by 2
1 more
  1. 2024On the Hallucination in Simultaneous Machine Translationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2022Unifying the Convergences in Multilingual Neural Machine Translation
  2. 2021Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy
  3. 2025SeqPO-SiMT: Sequential Policy Optimization for Simultaneous Machine Translation
  4. 2025Multi-Hypothesis Distillation of Multilingual Neural Translation Models for Low-Resource Languages
  5. 2025SimulPL: Aligning Human Preferences in Simultaneous Machine Translation
  6. 2021Selective Knowledge Distillation for Neural Machine Translation
  7. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation
  8. 2023An Empirical Study of Leveraging Knowledge Distillation for Compressing Multilingual Neural Machine Translation Models