# LibriS2S: A German-English Speech-to-Speech Translation Corpus

arXiv: https://arxiv.org/abs/2204.10593 · HTML: https://mlsanity.com/paper/2204.10593/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 22 Apr 2022.

## Builds on (most cited first, 15 of 15)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron](https://mlsanity.com/paper/1803.09047/) (2018)
- [MOSNet: Deep Learning based Objective Assessment for Voice Conversion](https://mlsanity.com/paper/1904.08352/) (2019)
- [Speech Translation and the End-to-End Promise: Taking Stock of Where We Are](https://mlsanity.com/paper/2004.06358/) (2020)
- [Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech](https://mlsanity.com/paper/2005.05106/) (2020)
- [MultiSpeech: Multi-Speaker Text to Speech with Transformer](https://mlsanity.com/paper/2006.04664/) (2020)
- [Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation](https://mlsanity.com/paper/1802.03142/) (2018)
- [MaSS: A Large and Clean Multilingual Corpus of Sentence-aligned Spoken Utterances Extracted from the Bible](https://mlsanity.com/paper/1907.12895/) (2019)
- [Effect of data reduction on sequence-to-sequence neural TTS](https://mlsanity.com/paper/1811.06315/) (2018)
- [LibriVoxDeEn: A Corpus for German-to-English Speech Translation and German Speech Recognition](https://mlsanity.com/paper/1910.07924/) (2019)

## Built on by (most cited first, 1 of 1)

- [Enhancing Speech-to-Speech Translation with Multiple TTS Targets](https://mlsanity.com/paper/2304.04618/) (2023)

## Abstract (arXiv)

Recently, we have seen an increasing interest in the area of speech-to-text translation. This has led to astonishing improvements in this area. In contrast, the activities in the area of speech-to-speech translation is still limited, although it is essential to overcome the language barrier. We believe that one of the limiting factors is the availability of appropriate training data. We address this issue by creating LibriS2S, to our knowledge the first publicly available speech-to-speech training corpus between German and English. For this corpus, we used independently created audio for German and English leading to an unbiased pronunciation of the text in both languages. This allows the creation of a new text-to-speech and speech-to-speech translation model that directly learns to generate the speech signal based on the pronunciation of the source language. Using this created corpus, we propose Text-to-Speech models based on the example of the recently proposed FastSpeech 2 model that integrates source language information. We do this by adapting the model to take information such as the pitch, energy or transcript from the source speech as additional input.