# Improving Semi-supervised End-to-end Automatic Speech Recognition using CycleGAN and Inter-domain Losses

arXiv: https://arxiv.org/abs/2210.11642 · HTML: https://mlsanity.com/paper/2210.11642/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 20 Oct 2022.

## Builds on (most cited first, 22 of 22)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Unsupervised Machine Translation Using Monolingual Corpora Only](https://mlsanity.com/paper/1711.00043/) (2017)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [End-to-End Attention-based Large Vocabulary Speech Recognition](https://mlsanity.com/paper/1508.04395/) (2015)
- [Unsupervised Neural Machine Translation](https://mlsanity.com/paper/1710.11041/) (2017)
- [EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding](https://mlsanity.com/paper/1507.08240/) (2015)
- [End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results](https://mlsanity.com/paper/1412.1602/) (2014)
- [Listening while Speaking: Speech Chain by Deep Learning](https://mlsanity.com/paper/1707.04879/) (2017)
- [Very Deep Convolutional Networks for End-to-End Speech Recognition](https://mlsanity.com/paper/1610.03022/) (2016)
- [Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition](https://mlsanity.com/paper/1610.09975/) (2016)
- [Cycle-consistency training for end-to-end speech recognition](https://mlsanity.com/paper/1811.01690/) (2018)
- [Back-Translation-Style Data Augmentation for End-to-End ASR](https://mlsanity.com/paper/1807.10893/) (2018)
- [Semi-supervised Sequence-to-sequence ASR using Unpaired Speech and Text](https://mlsanity.com/paper/1905.01152/) (2019)
- [Pretraining by Backtranslation for End-to-end ASR in Low-Resource Settings](https://mlsanity.com/paper/1812.03919/) (2018)

## Built on by (most cited first, 1 of 1)

- [Improving noisy student training for low-resource languages in End-to-End ASR using CycleGAN and inter-domain losses](https://mlsanity.com/paper/2407.21061/) (2024)

## Abstract (arXiv)

We propose a novel method that combines CycleGAN and inter-domain losses for semi-supervised end-to-end automatic speech recognition. Inter-domain loss targets the extraction of an intermediate shared representation of speech and text inputs using a shared network. CycleGAN uses cycle-consistent loss and the identity mapping loss to preserve relevant characteristics of the input feature after converting from one domain to another. As such, both approaches are suitable to train end-to-end models on unpaired speech-text inputs. In this paper, we exploit the advantages from both inter-domain loss and CycleGAN to achieve better shared representation of unpaired speech and text inputs and thus improve the speech-to-text mapping. Our experimental results on the WSJ eval92 and Voxforge (non English) show 8~8.5% character error rate reduction over the baseline, and the results on LibriSpeech test_clean also show noticeable improvement.