# Simultaneous Masking, Not Prompting Optimization: A Paradigm Shift in Fine-tuning LLMs for Simultaneous Translation

arXiv: https://arxiv.org/abs/2405.10443 · HTML: https://mlsanity.com/paper/2405.10443/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 16 May 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [Prompting PaLM for Translation: Assessing Strategies and Performance](https://mlsanity.com/paper/2211.09102/) (2022)
- [Learning to Translate in Real-time with Neural Machine Translation](https://mlsanity.com/paper/1610.00388/) (2016)
- [SimulEval: An Evaluation Toolkit for Simultaneous Translation](https://mlsanity.com/paper/2007.16193/) (2020)
- [Can neural machine translation do simultaneous translation?](https://mlsanity.com/paper/1606.02012/) (2016)
- [SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation](https://mlsanity.com/paper/2011.02048/) (2020)
- [Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation](https://mlsanity.com/paper/2206.05807/) (2022)
- [Simpler and Faster Learning of Adaptive Policies for Simultaneous Translation](https://mlsanity.com/paper/1909.01559/) (2019)
- [TransLLaMa: LLM-based Simultaneous Translation System](https://mlsanity.com/paper/2402.04636/) (2024)
- [Does Simultaneous Speech Translation need Simultaneous Models?](https://mlsanity.com/paper/2204.03783/) (2022)
- [SiLLM: Large Language Models for Simultaneous Machine Translation](https://mlsanity.com/paper/2402.13036/) (2024)

## Built on by (most cited first, 11 of 11)

- [StreamingThinker: Large Language Models Can Think While Reading](https://mlsanity.com/paper/2510.17238/) (2025)
- [Efficient and Adaptive Simultaneous Speech Translation with Fully Unidirectional Architecture](https://mlsanity.com/paper/2504.11809/) (2025)
- [LLM as Effective Streaming Processor: Bridging Streaming-Batch Mismatches with Group Position Encoding](https://mlsanity.com/paper/2505.16983/) (2025)
- [InfiniSST: Simultaneous Translation of Unbounded Speech with Large Language Model](https://mlsanity.com/paper/2503.02969/) (2025)
- [FASST: Fast LLM-based Simultaneous Speech Translation](https://mlsanity.com/paper/2408.09430/) (2024)
- [LLMs Can Achieve High-quality Simultaneous Machine Translation as Efficiently as Offline](https://mlsanity.com/paper/2504.09570/) (2025)
- [From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Models](https://mlsanity.com/paper/2603.04592/) (2026)
- [What Are They Filtering Out? An Experimental Benchmark of Filtering Strategies for Harm Reduction in Pretraining Datasets](https://mlsanity.com/paper/2503.05721/) (2025)
- [BeaverTalk: Oregon State University's IWSLT 2025 Simultaneous Speech Translation System](https://mlsanity.com/paper/2505.24016/) (2025)
- [AdaSR: Adaptive Streaming Reasoning with Hierarchical Relative Policy Optimization](https://mlsanity.com/paper/2606.14694/) (2026)
- [MLLP-VRAIN UPV system for the IWSLT 2026 Simultaneous Speech Translation task](https://mlsanity.com/paper/2606.17255/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have achieved state-of-the-art performance in various language processing tasks, motivating their adoption in simultaneous translation. Current fine-tuning methods to adapt LLMs for simultaneous translation focus on prompting optimization strategies using either data augmentation or prompt structure modifications. However, these methods suffer from several issues, such as unnecessarily expanded training sets, computational inefficiency from dumping the key and value cache, increased prompt sizes, or restriction to a single decision policy. To eliminate these issues, in this work, we propose SimulMask, a new paradigm for fine-tuning LLMs for simultaneous translation. It utilizes a novel attention mask approach that models simultaneous translation during fine-tuning by masking attention for a desired decision policy. Applying the proposed SimulMask on a Falcon LLM for the IWSLT 2017 dataset, we have observed a significant translation quality improvement compared to state-of-the-art prompting optimization strategies on five language pairs while reducing the computational cost.