mlsanityData to 14 Oct 2026

End-to-End Spoken Language Understanding: Performance analyses of a voice command task in a low resource setting

arXiv 2207.08179 ↗First posted 17 Jul 2022cs.CL cs.SD eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

Spoken Language Understanding (SLU) is a core task in most human-machine interaction systems. With the emergence of smart homes, smart phones and smart speakers, SLU has become a key technology for the industry. In a classical SLU approach, an Automatic Speech Recognition (ASR) module transcribes the speech signal into a textual representation from which a Natural Language Understanding (NLU) module extracts semantic information. Recently End-to-End SLU (E2E SLU) based on Deep Neural Networks has gained momentum since it benefits from the joint optimization of the ASR and the NLU parts, hence limiting the cascade of error effect of the pipeline architecture. However, little is known about the actual linguistic properties used by E2E models to predict concepts and intents from speech input. In this paper, we present a study identifying the signal features and other linguistic properties used by an E2E model to perform the SLU task. The study is carried out in the application domain of a smart home that has to handle non-English (here French) voice commands. The results show that a good E2E SLU performance does not always require a perfect ASR capability. Furthermore, the results show the superior capabilities of the E2E model in handling background noise and syntactic variation compared to the pipeline model. Finally, a finer-grained analysis suggests that the E2E model uses the pitch information of the input signal to identify voice command concepts. The results and methodology outlined in this paper provide a springboard for further analyses of E2E models in speech processing.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 24 of its 75 references that are in the map. First posted on arXiv 17 Jul 2022.

Builds on

24 of 75 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  3. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  4. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  5. 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 451
  6. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  7. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  8. 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 193
  9. 2016Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Fillingcited by 137
  10. 2018Towards end-to-end spoken language understandingcited by 92
  11. 2018Sequence-to-Sequence Data Augmentation for Dialogue Language Understandingcited by 68
  12. 2018Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modelingcited by 34
12 more
  1. 2019Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portabilitycited by 29
  2. 2017Towards End-to-End Speech Recognition with Deep Convolutional Neural Networkscited by 24
  3. 2019Using Speech Synthesis to Train End-to-End Spoken Language Understanding Modelscited by 24
  4. 2020Speech To Semantics: Improve ASR and NLU Jointly via All-Neural Interfacescited by 24
  5. 2018Training Neural Speech Recognition Systems with Synthetic Speech Augmentationcited by 21
  6. 2020Pretrained Semantic Speech Embeddings for End-to-End Spoken Language Understanding via Cross-Modal Teacher-Student Learningcited by 14
  7. 2014Parallel training of DNNs with Natural Gradient and Parameter Averagingcited by 11
  8. 2017Sequential Dialogue Context Modeling for Spoken Language Understandingcited by 11
  9. 2018End-to-end named entity extraction from speechcited by 11
  10. 2020Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understandingcited by 8
  11. 2017Improving Slot Filling Performance with Attentive Neural Networks on Dependency Structurescited by 7
  12. 2017ASR error management for improving spoken language understandingcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 3.

3 papers.

  1. 2023LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speechcited by 18
  2. 2023Modality Confidence Aware Training for Robust End-to-End Spoken Language Understandingcited by 2
  3. 2023Spoken Dialogue System for Medical Prescription Acquisition on Smartphone: Development, Corpus and Evaluationcited by 0