# Word Order Does Not Matter For Speech Recognition

arXiv: https://arxiv.org/abs/2110.05994 · HTML: https://mlsanity.com/paper/2110.05994/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 36 references that are in the map.
- First posted on arXiv 12 Oct 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Reducing Transformer Depth on Demand with Structured Dropout](https://mlsanity.com/paper/1909.11556/) (2019)
- [TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech](https://mlsanity.com/paper/2007.06028/) (2020)
- [Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2010.10504/) (2020)
- [An Unsupervised Autoregressive Model for Speech Representation Learning](https://mlsanity.com/paper/1904.03240/) (2019)
- [Unsupervised Speech Recognition](https://mlsanity.com/paper/2105.11084/) (2021)
- [Improved Noisy Student Training for Automatic Speech Recognition](https://mlsanity.com/paper/2005.09629/) (2020)
- [Multi-task self-supervised learning for Robust Speech Recognition](https://mlsanity.com/paper/2001.09239/) (2020)
- [Self-Training for End-to-End Speech Recognition](https://mlsanity.com/paper/1909.09116/) (2019)
- [End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures](https://mlsanity.com/paper/1911.08460/) (2019)
- [Iterative Pseudo-Labeling for Speech Recognition](https://mlsanity.com/paper/2005.09267/) (2020)
- [Rethinking Evaluation in ASR: Are Our Models Robust Enough?](https://mlsanity.com/paper/2010.11745/) (2020)
- [SlimIPL: Language-Model-Free Iterative Pseudo-Labeling](https://mlsanity.com/paper/2010.11524/) (2020)
- [Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependencies](https://mlsanity.com/paper/2011.00406/) (2020)
- [wav2letter++: The Fastest Open-source Speech Recognition System](https://mlsanity.com/paper/1812.07625/) (2018)
- [Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition](https://mlsanity.com/paper/2106.08922/) (2021)
- [Semi-supervised ASR by End-to-end Self-training](https://mlsanity.com/paper/2001.09128/) (2020)
- [Wav2vec-C: A Self-supervised Model for Speech Representation Learning](https://mlsanity.com/paper/2103.08393/) (2021)
- [Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddings](https://mlsanity.com/paper/1804.00316/) (2018)
- [Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matching](https://mlsanity.com/paper/1812.09323/) (2018)
- [Joint Masked CPC and CTC Training for ASR](https://mlsanity.com/paper/2011.00093/) (2020)
- [Semi-Supervised Speech Recognition via Graph-based Temporal Classification](https://mlsanity.com/paper/2010.15653/) (2020)
- [Visually grounded learning of keyword prediction from untranscribed speech](https://mlsanity.com/paper/1703.08136/) (2017)
- [Acoustically Grounded Word Embeddings for Improved Acoustics-to-Word Speech Recognition](https://mlsanity.com/paper/1903.12306/) (2019)
- [Completely Unsupervised Speech Recognition By A Generative Adversarial Network Harmonized With Iteratively Refined Hidden Markov Models](https://mlsanity.com/paper/1904.04100/) (2019)
- [Large scale weakly and semi-supervised learning for low-resource video ASR](https://mlsanity.com/paper/2005.07850/) (2020)

## Built on by (most cited first, 2 of 2)

- [Star Temporal Classification: Sequence Classification with Partially Labeled Data](https://mlsanity.com/paper/2201.12208/) (2022)
- [Audio-to-Intent Using Acoustic-Textual Subword Representations from End-to-End ASR](https://mlsanity.com/paper/2210.12134/) (2022)

## Abstract (arXiv)

In this paper, we study training of automatic speech recognition system in a weakly supervised setting where the order of words in transcript labels of the audio training data is not known. We train a word-level acoustic model which aggregates the distribution of all output frames using LogSumExp operation and uses a cross-entropy loss to match with the ground-truth words distribution. Using the pseudo-labels generated from this model on the training set, we then train a letter-based acoustic model using Connectionist Temporal Classification loss. Our system achieves 2.3%/4.6% on test-clean/test-other subsets of LibriSpeech, which closely matches with the supervised baseline's performance.