mlsanityData to 14 Oct 2026

Speech Model Pre-training for End-to-End Spoken Language Understanding

arXiv 1904.03670 ↗First posted 7 Apr 2019eess.AS cs.CL cs.LGcited by 193

Not on the map. Why?

Abstract (arXiv)

Whereas conventional spoken language understanding (SLU) systems map speech to text, and then text to intent, end-to-end SLU systems map speech directly to intent through a single trainable model. Achieving high accuracy with these end-to-end models without a large amount of training data is difficult. We propose a method to reduce the data requirements of end-to-end SLU in which the model is first pre-trained to predict words and phonemes, thus learning good features for SLU. We introduce a new SLU dataset, Fluent Speech Commands, and show that our method improves performance both when the full dataset is used for training and when only a small subset is used. We also describe preliminary experiments to gauge the model's ability to generalize to new phrases not heard during training.

Key facts. Cited by 193 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 45; 2022: 44; 2023: 38; 2024: 28; 2025: 18; 2026: 20.

193 papers.

Sort:

By area:

  1. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  2. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 406
  3. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  4. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  5. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
  6. 2021Decoupled Contrastive Learningcited by 143
  7. 2023BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writingcited by 79
  8. 2021SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speechcited by 72
  9. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
  10. 2023What Do Self-Supervised Speech Models Know About Words?cited by 64
  11. 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 40
  12. 2022End-to-end Spoken Conversational Question Answering: Task, Dataset and Modelcited by 37
38 more
  1. 2022SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Taskscited by 35
  2. 2023SpeechPrompt v2: Prompt Tuning for Speech Classification Taskscited by 32
  3. 2023Structured Pruning of Self-Supervised Pre-trained Models for Speech Recognition and Understandingcited by 31
  4. 2023Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?cited by 28
  5. 2021Speech-language Pre-training for End-to-end Spoken Language Understandingcited by 27
  6. 2025Streaming Sequence-to-Sequence Learning with Delayed Streams Modelingcited by 27
  7. 2021Phone-to-audio alignment without text: A Semi-supervised Approachcited by 26
  8. 2022WAVPROMPT: Towards Few-Shot Spoken Language Understanding with Frozen Language Modelscited by 26
  9. 2021Learning Efficient Representations for Keyword Spotting with Triplet Losscited by 24
  10. 2021Integration of Pre-trained Networks with Continuous Token Interface for End-to-End Spoken Language Understandingcited by 22
  11. 2022STOP: A dataset for Spoken Task Oriented Semantic Parsingcited by 22
  12. 2023Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Computecited by 21
  13. 2021Don't speak too fast: The impact of data bias on self-supervised speech modelscited by 20
  14. 2022Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systemscited by 20
  15. 2022SpeechPrompt: An Exploration of Prompt Tuning on Generative Spoken Language Model for Speech Processing Taskscited by 19
  16. 2022A Study on the Integration of Pre-trained SSL, ASR, LM and SLU Models for Spoken Language Understandingcited by 18
  17. 2023LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speechcited by 18
  18. 2021End-to-End Spoken Language Understanding for Generalized Voice Assistantscited by 17
  19. 2022Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptationcited by 17
  20. 2023Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Headscited by 17
  21. 2021Revisiting the Boundary between ASR and NLU in the Age of Conversational Dialog Systemscited by 15
  22. 2021Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classificationcited by 14
  23. 2022SoundChoice: Grapheme-to-Phoneme Models with Semantic Disambiguationcited by 14
  24. 2021Did the Model Change? Efficiently Assessing Machine Learning API Shiftscited by 13
  25. 2021On the Use of External Data for Spoken Named Entity Recognitioncited by 12
  26. 2024SpeechPrompt: Prompting Speech Language Models for Speech Processing Taskscited by 12
  27. 2021Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classificationcited by 11
  28. 2021Timers and Such: A Practical Benchmark for Spoken Language Understanding with Numberscited by 11
  29. 2021Rethinking End-to-End Evaluation of Decomposable Tasks: A Case Study on Spoken Language Understandingcited by 11
  30. 2022End-to-end model for named entity recognition from speech without paired training datacited by 11
  31. 2022Blockwise Streaming Transformer for Spoken Language Understanding and Simultaneous Speech Translationcited by 11
  32. 2023ITALIC: An Italian Intent Classification Datasetcited by 10
  33. 2023Ecosystem-level Analysis of Deployed Machine Learning Reveals Homogeneous Outcomescited by 10
  34. 2023FlowMur: A Stealthy and Practical Audio Backdoor Attack with Limited Knowledgecited by 10
  35. 2025GLAP: General contrastive audio-text pretraining across domains and languagescited by 10
  36. 2021Do as I mean, not as I say: Sequence Loss Training for Spoken Language Understandingcited by 9
  37. 2021Leveraging Pre-trained Language Model for Speech Sentiment Analysiscited by 9
  38. 2021Integrating Dialog History into End-to-End Spoken Language Understanding Systemscited by 9

50 most cited of 193 All 193 (JSON)