mlsanityData to 7 Oct 2026

Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding

arXiv 2205.00693 ↗First posted 2 May 2022cs.CL cs.SD eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

Spoken language understanding (SLU) is an essential task for machines to understand human speech for better interactions. However, errors from the automatic speech recognizer (ASR) usually hurt the understanding performance. In reality, ASR systems may not be easy to adjust for the target scenarios. Therefore, this paper focuses on learning utterance representations that are robust to ASR errors using a contrastive objective, and further strengthens the generalization ability by combining supervised contrastive learning and self-distillation in model fine-tuning. Experiments on three benchmark datasets demonstrate the effectiveness of our proposed approach.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 25 of its 31 references that are in the map. First posted on arXiv 2 May 2022.

Builds on

25 of 31 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2020Supervised Contrastive Learningcited by 2,803
  6. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  7. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  8. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  9. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
  10. 2020Supervised Contrastive Learning for Pre-trained Language Model Fine-tuningcited by 275
  11. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 249
  12. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 197
13 more
  1. 2020Self-Distillation Amplifies Regularization in Hilbert Spacecited by 171
  2. 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
  3. 2020Regularizing Class-wise Predictions via Self-knowledge Distillationcited by 144
  4. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
  5. 2021ReSSL: Relational Self-Supervised Learning with Weak Augmentationcited by 83
  6. 2020ASR Error Correction and Domain Adaptation Using Machine Translationcited by 48
  7. 2020End-to-End Neural Transformer Based Spoken Language Understandingcited by 31
  8. 2021Phoneme-BERT: Joint Language Modelling of Phoneme Sequence and ASR Transcriptcited by 21
  9. 2022Error Correction in ASR using Sequence-to-Sequence Modelscited by 19
  10. 2021Similarity Contrastive Estimation for Self-Supervised Soft Contrastive Learningcited by 18
  11. 2020Adapting Pretrained Transformer to Lattices for Spoken Language Understandingcited by 12
  12. 2021N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypothesescited by 12
  13. 2020Learning Spoken Language Representations with Neural Lattice Language Modelingcited by 6

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 1.

4 papers.

  1. 2023ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understandingcited by 6
  2. 2023The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech Translationcited by 4
  3. 2022Improve Text Classification Accuracy with Intent Informationcited by 0
  4. 2024Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learningcited by 0