# StreamUni: Achieving Streaming Speech Translation with a Unified Large Speech-Language Model

arXiv: https://arxiv.org/abs/2507.07803 · HTML: https://mlsanity.com/paper/2507.07803/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 10 Jul 2025.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities](https://mlsanity.com/paper/2305.11000/) (2023)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://mlsanity.com/paper/2312.05187/) (2023)
- [Direct speech-to-speech translation with discrete units](https://mlsanity.com/paper/2107.05604/) (2021)
- [SimulEval: An Evaluation Toolkit for Simultaneous Translation](https://mlsanity.com/paper/2007.16193/) (2020)
- [SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation](https://mlsanity.com/paper/2011.02048/) (2020)
- [Attention as a Guide for Simultaneous Speech Translation](https://mlsanity.com/paper/2212.07850/) (2022)
- [Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR](https://mlsanity.com/paper/2106.06636/) (2021)
- [Learning When to Translate for Streaming Speech](https://mlsanity.com/paper/2109.07368/) (2021)
- [Speech Translation with Large Language Models: An Industrial Practice](https://mlsanity.com/paper/2312.13585/) (2023)
- [Towards Achieving Human Parity on End-to-end Simultaneous Speech Translation via LLM Agent](https://mlsanity.com/paper/2407.21646/) (2024)
- [Sentence Boundary Augmentation For Neural Machine Translation Robustness](https://mlsanity.com/paper/2010.11132/) (2020)
- [Large Language Models Are Read/Write Policy-Makers for Simultaneous Generation](https://mlsanity.com/paper/2501.00868/) (2025)
- [Overcoming Non-monotonicity in Transducer-based Streaming Generation](https://mlsanity.com/paper/2411.17170/) (2024)

## Built on by (most cited first, 6 of 6)

- [SASST: Leveraging Syntax-Aware Chunking and LLMs for Simultaneous Speech Translation](https://mlsanity.com/paper/2508.07781/) (2025)
- [Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech](https://mlsanity.com/paper/2604.21045/) (2026)
- [DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Translation with SpeechLLMs](https://mlsanity.com/paper/2605.31432/) (2026)
- [OmniFusion: Simultaneous Multilingual Multimodal Translations via Modular Fusion](https://mlsanity.com/paper/2512.00234/) (2025)
- [RASST: Retrieval-Augmented Simultaneous Speech Translation](https://mlsanity.com/paper/2601.22777/) (2026)
- [Dual-Stream Simultaneous Translation via 2D Grid Attention](https://mlsanity.com/paper/2609.34902/) (2026)

## Abstract (arXiv)

Streaming speech translation (StreamST) requires determining appropriate timing, known as policy, to generate translations while continuously receiving source speech inputs, balancing low latency with high translation quality. However, existing StreamST methods typically operate on sentence-level speech segments, referred to as simultaneous speech translation (SimulST). In practice, they require collaboration with segmentation models to accomplish StreamST, where the truncated speech segments constrain SimulST models to make policy decisions and generate translations based on limited contextual information. Moreover, SimulST models struggle to learn effective policies due to the complexity of speech inputs and cross-lingual generation. To address these challenges, we propose StreamUni, which achieves StreamST through a unified Large Speech-Language Model (LSLM). Specifically, StreamUni incorporates speech Chain-of-Thought (CoT) in guiding the LSLM to generate multi-stage outputs. Leveraging these multi-stage outputs, StreamUni simultaneously accomplishes speech segmentation, policy decision, and translation generation, completing StreamST without requiring massive policy-specific training. Additionally, we propose a streaming CoT training method that enhances low-latency policy decisions and generation capabilities using limited CoT data. Experiments demonstrate that our approach achieves state-of-the-art performance on StreamST tasks.