# Exploring Phoneme-Level Speech Representations for End-to-End Speech Translation

arXiv: https://arxiv.org/abs/1906.01199 · HTML: https://mlsanity.com/paper/1906.01199/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 15 of 15)

- [The Multilingual TEDx Corpus for Speech Recognition and Translation](https://mlsanity.com/paper/2102.01757/) (2021)
- [CTC-based Compression for Direct Speech Translation](https://mlsanity.com/paper/2102.01578/) (2021)
- [Large-Scale Self- and Semi-Supervised Learning for Speech Translation](https://mlsanity.com/paper/2104.06678/) (2021)
- [Revisiting End-to-End Speech-to-Text Translation From Scratch](https://mlsanity.com/paper/2206.04571/) (2022)
- [Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks](https://mlsanity.com/paper/2105.00573/) (2021)
- [Speechformer: Reducing Information Loss in Direct Speech Translation](https://mlsanity.com/paper/2109.04574/) (2021)
- [On-the-Fly Aligned Data Augmentation for Sequence-to-Sequence ASR](https://mlsanity.com/paper/2104.01393/) (2021)
- [Pre-training for Spoken Language Understanding with Joint Textual and Phonetic Representation Learning](https://mlsanity.com/paper/2104.10357/) (2021)
- [AdaTranS: Adapting with Boundary-based Shrinking for End-to-End Speech Translation](https://mlsanity.com/paper/2212.08911/) (2022)
- [AlloST: Low-resource Speech Translation without Source Transcription](https://mlsanity.com/paper/2105.00171/) (2021)
- [Bridging the Granularity Gap for Acoustic Modeling](https://mlsanity.com/paper/2305.17356/) (2023)
- [Multiformer: A Head-Configurable Transformer-Based Model for Direct Speech Translation](https://mlsanity.com/paper/2205.07100/) (2022)
- [Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation](https://mlsanity.com/paper/2302.10871/) (2023)
- [How do Hyenas deal with Human Speech? Speech Recognition and Translation with ConfHyena](https://mlsanity.com/paper/2402.13208/) (2024)
- [Joint Training And Decoding for Multilingual End-to-End Simultaneous Speech Translation](https://mlsanity.com/paper/2503.11080/) (2025)

## Abstract (arXiv)

Previous work on end-to-end translation from speech has primarily used frame-level features as speech representations, which creates longer, sparser sequences than text. We show that a naive method to create compressed phoneme-like speech representations is far more effective and efficient for translation than traditional frame-level speech features. Specifically, we generate phoneme labels for speech frames and average consecutive frames with the same label to create shorter, higher-level source sequences for translation. We see improvements of up to 5 BLEU on both our high and low resource language pairs, with a reduction in training time of 60%. Our improvements hold across multiple data sizes and two language pairs.