mlsanityData to 7 Oct 2026

Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation

arXiv 1802.03142 ↗First posted 9 Feb 2018cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

Recent works in spoken language translation (SLT) have attempted to build end-to-end speech-to-text translation without using source language transcription during learning or decoding. However, while large quantities of parallel texts (such as Europarl, OpenSubtitles) are available for training machine translation systems, there are no large (100h) and open source parallel corpora that include speech in a source language aligned to text in a target language. This paper tries to fill this gap by augmenting an existing (monolingual) corpus: LibriSpeech. This corpus, used for automatic speech recognition, is derived from read audiobooks from the LibriVox project, and has been carefully segmented and aligned. After gathering French e-books corresponding to the English audio-books from LibriSpeech, we align speech segments at the sentence level with their respective translations and obtain 236h of usable parallel data. This paper presents the details of the processing as well as a manual evaluation conducted on a small subset of the corpus. This evaluation shows that the automatic alignments scores are reasonably correlated with the human judgments of the bilingual alignment quality. We believe that this corpus (which is made available online) is useful for replicable experiments in direct speech translation or more general spoken language translation experiments.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 11; 2023: 8; 2024: 3; 2025: 0; 2026: 1.

34 papers.

Sort:

  1. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  2. 2021The Multilingual TEDx Corpus for Speech Recognition and Translationcited by 101
  3. 2021Learning Shared Semantic Space for Speech-to-Text Translationcited by 49
  4. 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
  5. 2021Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translationcited by 47
  6. 2021End-to-end Speech Translation via Cross-modal Progressive Trainingcited by 47
  7. 2023Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalizationcited by 43
  8. 2021RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformercited by 39
  9. 2023MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translationcited by 39
  10. 2022Revisiting End-to-End Speech-to-Text Translation From Scratchcited by 26
  11. 2021BSTC: A Large-Scale Chinese-English Speech Translation Datasetcited by 16
  12. 2022Non-Parametric Domain Adaptation for End-to-End Speech Translationcited by 12
22 more
  1. 2022Simple and Effective Unsupervised Speech Translationcited by 12
  2. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  3. 2021Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediatescited by 9
  4. 2022Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Datacited by 9
  5. 2023End-to-End Speech-to-Text Translation: A Surveycited by 9
  6. 2022ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understandingcited by 8
  7. 2021Kosp2e: Korean Speech to English Translation Corpuscited by 7
  8. 2021SpliceOut: A Simple and Efficient Audio Augmentation Methodcited by 7
  9. 2023Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learningcited by 7
  10. 2022On the Impact of Noises in Crowd-Sourced Data for Speech Translationcited by 6
  11. 2022Does Joint Training Really Help Cascaded Speech Translation?cited by 5
  12. 2023HK-LegiCoST: Leveraging Non-Verbatim Transcripts for Speech Translationcited by 3
  13. 2024FFSTC: Fongbe to French Speech Translation Corpuscited by 3
  14. 2022A High-Quality and Large-Scale Dataset for English-Vietnamese Speech Translationcited by 2
  15. 2023End-to-End Single-Channel Speaker-Turn Aware Conversational Speech Translationcited by 2
  16. 2022LibriS2S: A German-English Speech-to-Speech Translation Corpuscited by 1
  17. 2023DropDim: A Regularization Method for Transformer Networkscited by 1
  18. 2024Conditioning LLMs with Emotion in Neural Machine Translationcited by 1
  19. 2022End-to-End Speech Translation of Arabic to English Broadcast Newscited by 0
  20. 2023Selective Data Augmentation for Robust Speech Translationcited by 0
  21. 2024Usefulness of Emotional Prosody in Neural Machine Translationcited by 0
  22. 2026English to Central Kurdish Speech Translation: Corpus Creation, Evaluation, and Orthographic Standardizationcited by 0