# Speech Segmentation Optimization using Segmented Bilingual Speech Corpus for End-to-end Speech Translation

arXiv: https://arxiv.org/abs/2203.15479 · HTML: https://mlsanity.com/paper/2203.15479/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 27 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [ESPnet-ST: All-in-One Speech Translation Toolkit](https://mlsanity.com/paper/2004.10234/) (2020)
- [End-to-End Automatic Speech Recognition Integrated With CTC-Based Voice Activity Detection](https://mlsanity.com/paper/2002.00551/) (2020)
- [Beyond Voice Activity Detection: Hybrid Audio Segmentation for Direct Speech Translation](https://mlsanity.com/paper/2104.11710/) (2021)
- [End-to-End Speech Translation with Pre-trained Models and Adapters: UPC at IWSLT 2021](https://mlsanity.com/paper/2105.04512/) (2021)
- [Segmenting Subtitles for Correcting ASR Segmentation Errors](https://mlsanity.com/paper/2104.07868/) (2021)

## Built on by (most cited first, 7 of 7)

- [How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?](https://mlsanity.com/paper/2412.18495/) (2024)
- [Long-Form End-to-End Speech Translation via Latent Alignment Segmentation](https://mlsanity.com/paper/2309.11384/) (2023)
- [Long-form Simultaneous Speech Translation: Thesis Proposal](https://mlsanity.com/paper/2310.11141/) (2023)
- [How to Evaluate Speech Translation with Source-Aware Neural MT Metrics](https://mlsanity.com/paper/2511.03295/) (2025)
- [Lightweight Audio Segmentation for Long-form Speech Translation](https://mlsanity.com/paper/2406.10549/) (2024)
- [Overcoming Latency Bottlenecks in On-Device Speech Translation: A Cascaded Approach with Alignment-Based Streaming MT](https://mlsanity.com/paper/2508.13358/) (2025)
- [DPO-Tuned Large Language Models for Segmentation in Simultaneous Speech Translation](https://mlsanity.com/paper/2510.12195/) (2025)

## Abstract (arXiv)

Speech segmentation, which splits long speech into short segments, is essential for speech translation (ST). Popular VAD tools like WebRTC VAD have generally relied on pause-based segmentation. Unfortunately, pauses in speech do not necessarily match sentence boundaries, and sentences can be connected by a very short pause that is difficult to detect by VAD. In this study, we propose a speech segmentation method using a binary classification model trained using a segmented bilingual speech corpus. We also propose a hybrid method that combines VAD and the above speech segmentation method. Experimental results revealed that the proposed method is more suitable for cascade and end-to-end ST systems than conventional segmentation methods. The hybrid approach further improved the translation performance.