mlsanityData to 14 Oct 2026

Do We Still Need Automatic Speech Recognition for Spoken Language Understanding?

arXiv 2111.14842 ↗First posted 29 Nov 2021eess.AS cs.CL cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Spoken language understanding (SLU) tasks are usually solved by first transcribing an utterance with automatic speech recognition (ASR) and then feeding the output to a text-based model. Recent advances in self-supervised representation learning for speech data have focused on improving the ASR component. We investigate whether representation learning for speech has matured enough to replace ASR in SLU. We compare learned speech features from wav2vec 2.0, state-of-the-art ASR transcripts, and the ground truth text as input for a novel speech-based named entity recognition task, a cardiac arrest detection task on real-world emergency calls and two existing SLU benchmarks. We show that learned speech features are superior to ASR transcripts on three classification tasks. For machine translation, ASR transcripts are still the better choice. We highlight the intrinsic robustness of wav2vec 2.0 representations to out-of-vocabulary words as key to better performance.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 15 of its 18 references that are in the map. First posted on arXiv 29 Nov 2021.

Builds on

15 of 18 references.

References · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  5. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  6. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  7. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  8. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 198
  9. 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 193
  10. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 151
  11. 2021Unsupervised Speech Recognitioncited by 148
  12. 2019Learning Problem-agnostic Speech Representations from Multiple Self-supervised Taskscited by 87
3 more
  1. 2020Semi-Supervised Spoken Language Understanding via Self-Supervised Speech and Language Model Pretrainingcited by 35
  2. 2021Speech-language Pre-training for End-to-end Spoken Language Understandingcited by 27
  3. 2021On Scaling Contrastive Representations for Low-Resource Speech Recognitioncited by 3

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2; 2023: 1.

5 papers.

  1. 2021On the Use of External Data for Spoken Named Entity Recognitioncited by 12
  2. 2022Bridging Speech and Textual Pre-trained Models with Unsupervised ASRcited by 3
  3. 2022Finstreder: Simple and fast Spoken Language Understanding with Finite State Transducers using modern Speech-to-Text modelscited by 2
  4. 2021Multi-Variant Consistency based Self-supervised Learning for Robust Automatic Speech Recognitioncited by 1
  5. 2023Sentence Embedder Guided Utterance Encoder (SEGUE) for Spoken Language Understandingcited by 1