mlsanityData to 7 Oct 2026

Leveraging Pretrained ASR Encoders for Effective and Efficient End-to-End Speech Intent Classification and Slot Filling

arXiv 2307.07057 ↗First posted 13 Jul 2023cs.CL cs.CV cs.SDcited by 3

Not on the map. Why?

Abstract (arXiv)

We study speech intent classification and slot filling (SICSF) by proposing to use an encoder pretrained on speech recognition (ASR) to initialize an end-to-end (E2E) Conformer-Transformer model, which achieves the new state-of-the-art results on the SLURP dataset, with 90.14% intent accuracy and 82.27% SLURP-F1. We compare our model with encoders pretrained on self-supervised learning (SSL), and show that ASR pretraining is much more effective than SSL for SICSF. To explore parameter efficiency, we freeze the encoder and add Adapter modules, and show that parameter efficiency is only achievable with an ASR-pretrained encoder, while the SSL encoder needs full finetuning to achieve comparable results. In addition, we provide an in-depth comparison on end-to-end models versus cascading models (ASR+NLU), and show that E2E models are better than cascaded models unless an oracle ASR model is provided. Last but not least, our model is the first E2E model that achieves the same performance as cascading models with oracle ASR. Code, checkpoints and configs are available.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 20 of its 24 references that are in the map. First posted on arXiv 13 Jul 2023.

Builds on

20 of 24 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  9. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  10. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 197
  11. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  12. 2022Contextual Adapters for Personalized Speech Recognition in Neural Transducerscited by 57
8 more
  1. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  2. 2020Continuous Speech Separation with Conformercited by 36
  3. 2020Knowledge Distillation for Improved Accuracy in Spoken Question Answeringcited by 31
  4. 2022Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognitioncited by 31
  5. 2021Integration of Pre-trained Networks with Continuous Token Interface for End-to-End Spoken Language Understandingcited by 22
  6. 2018ODSQA: Open-domain Spoken Question Answering Datasetcited by 21
  7. 2021Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classificationcited by 11
  8. 2021Leveraging Pre-trained Language Model for Speech Sentiment Analysiscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 1; 2026: 1.

3 papers.

  1. 2025Exploring Fine-Tuning of Large Audio Language Models for Spoken Language Understanding under Limited Speech Datacited by 6
  2. 2023Speech Understanding on Tiny Devices with A Learning Cachecited by 3
  3. 2026Selective Capability Unlearning in End-to-End Spoken Language Understandingcited by 1