# SpeeChain: A Speech Toolkit for Large-Scale Machine Speech Chain

arXiv: https://arxiv.org/abs/2301.02966 · HTML: https://mlsanity.com/paper/2301.02966/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 8 Jan 2023.

## Builds on (most cited first, 22 of 22)

- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [Self-Training for End-to-End Speech Recognition](https://mlsanity.com/paper/1909.09116/) (2019)
- [Speech Recognition with Augmented Synthesized Speech](https://mlsanity.com/paper/1909.11699/) (2019)
- [ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit](https://mlsanity.com/paper/1910.10909/) (2019)
- [Generating Synthetic Audio Data for Attention-Based Speech Recognition Systems](https://mlsanity.com/paper/1912.09257/) (2019)
- [Listening while Speaking: Speech Chain by Deep Learning](https://mlsanity.com/paper/1707.04879/) (2017)
- [MultiSpeech: Multi-Speaker Text to Speech with Transformer](https://mlsanity.com/paper/2006.04664/) (2020)
- [Almost Unsupervised Text to Speech and Automatic Speech Recognition](https://mlsanity.com/paper/1905.06791/) (2019)
- [Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions](https://mlsanity.com/paper/1904.02619/) (2019)
- [fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit](https://mlsanity.com/paper/2109.06912/) (2021)
- [Training Neural Speech Recognition Systems with Synthetic Speech Augmentation](https://mlsanity.com/paper/1811.00707/) (2018)
- [EAT: Enhanced ASR-TTS for Self-supervised Speech Recognition](https://mlsanity.com/paper/2104.07474/) (2021)
- [Machine Speech Chain with One-shot Speaker Adaptation](https://mlsanity.com/paper/1803.10525/) (2018)
- [PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit](https://mlsanity.com/paper/2205.12007/) (2022)
- [Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architectures](https://mlsanity.com/paper/2104.05379/) (2021)
- [Exploring Machine Speech Chain for Domain Adaptation and Few-Shot Speaker Adaptation](https://mlsanity.com/paper/2104.03815/) (2021)

## Built on by (most cited first, 2 of 2)

- [On Efficient Training of Large-Scale Deep Learning Models: A Literature Review](https://mlsanity.com/paper/2304.03589/) (2023)
- [Continual Learning in Machine Speech Chain Using Gradient Episodic Memory](https://mlsanity.com/paper/2411.18320/) (2024)

## Abstract (arXiv)

This paper introduces SpeeChain, an open-source Pytorch-based toolkit designed to develop the machine speech chain for large-scale use. This first release focuses on the TTS-to-ASR chain, a core component of the machine speech chain, that refers to the TTS data augmentation by unspoken text for ASR. To build an efficient pipeline for the large-scale TTS-to-ASR chain, we implement easy-to-use multi-GPU batch-level model inference, multi-dataloader batch generation, and on-the-fly data selection techniques. In this paper, we first explain the overall procedure of the TTS-to-ASR chain and the difficulties of each step. Then, we present a detailed ablation study on different types of unlabeled data, data filtering thresholds, batch composition, and real-synthetic data ratios. Our experimental results on train_clean_460 of LibriSpeech demonstrate that our TTS-to-ASR chain can significantly improve WER in a semi-supervised setting.