mlsanityData to 14 Oct 2026

Content-Context Factorized Representations for Automated Speech Recognition

arXiv 2205.09872 ↗First posted 19 May 2022eess.AS cs.LG cs.SDcited by 8

Not on the map. Why?

Abstract (arXiv)

Deep neural networks have largely demonstrated their ability to perform automated speech recognition (ASR) by extracting meaningful features from input audio frames. Such features, however, may consist not only of information about the spoken language content, but also may contain information about unnecessary contexts such as background noise and sounds or speaker identity, accent, or protected attributes. Such information can directly harm generalization performance, by introducing spurious correlations between the spoken words and the context in which such words were spoken. In this work, we introduce an unsupervised, encoder-agnostic method for factoring speech-encoder representations into explicit content-encoding representations and spurious context-encoding representations. By doing so, we demonstrate improved performance on standard ASR benchmarks, as well as improved performance in both real-world and artificially noisy ASR scenarios.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 19 May 2022.

Builds on

18 of 28 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  7. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  8. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  9. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  10. 2022Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Predictioncited by 287
  11. 2016Disentangling factors of variation in deep representations using adversarial trainingcited by 87
  12. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 64
6 more
  1. 2019Learning Disentangled Representations via Mutual Information Estimationcited by 52
  2. 2021Multi-Modal Pre-Training for Automated Speech Recognitioncited by 10
  3. 2020Intra-class variation reduction of speaker representation in disentanglement frameworkcited by 5
  4. 2021Scenario Aware Speech Recognition: Advancements for Apollo Fearless Steps & CHiME-4 Corporacited by 4
  5. 2021Disentanglement by Cyclic Reconstructioncited by 3
  6. 2021Using multiple reference audios and style embedding constraints for speech synthesiscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 5.

8 papers.

  1. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  2. 2023Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clusteringcited by 22
  3. 2023R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Piecescited by 7
  4. 2024Representation Purification for End-to-End Speech Translationcited by 3
  5. 2024Task Oriented Dialogue as a Catalyst for Self-Supervised Automatic Speech Recognitioncited by 1
  6. 2024Multi-Stage Multi-Modal Pre-Training for Automatic Speech Recognitioncited by 1
  7. 2024JOOCI: a Framework for Learning Comprehensive Speech Representationscited by 1
  8. 2024An Efficient Self-Learning Framework For Interactive Spoken Dialog Systemscited by 0