mlsanityData to 7 Oct 2026

Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification

arXiv 2102.07370 ↗First posted 15 Feb 2021cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Intent classification is a task in spoken language understanding. An intent classification system is usually implemented as a pipeline process, with a speech recognition module followed by text processing that classifies the intents. There are also studies of end-to-end system that takes acoustic features as input and classifies the intents directly. Such systems don't take advantage of relevant linguistic information, and suffer from limited training data. In this work, we propose a novel intent classification framework that employs acoustic features extracted from a pretrained speech recognition system and linguistic features learned from a pretrained language model. We use knowledge distillation technique to map the acoustic embeddings towards linguistic embeddings. We perform fusion of both acoustic and linguistic embeddings through cross-attention approach to classify intents. With the proposed method, we achieve 90.86% and 99.07% accuracy on ATIS and Fluent speech corpus, respectively.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 23 of its 31 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

23 of 31 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  7. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
  8. 2017Convolutional Sequence to Sequence Learningcited by 550
  9. 2015Listen, Attend and Spellcited by 542
  10. 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 449
  11. 2015Attention-Based Models for Speech Recognitioncited by 400
  12. 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 191
11 more
  1. 2018Towards end-to-end spoken language understandingcited by 92
  2. 2018From Audio to Semantics: Approaches to end-to-end spoken language understandingcited by 63
  3. 2020Leveraging Unpaired Text Data for Training End-to-End Speech-to-Intent Systemscited by 38
  4. 2016Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognitioncited by 33
  5. 2017Large-Scale Domain Adaptation via Teacher-Student Learningcited by 28
  6. 2019Using Speech Synthesis to Train End-to-End Spoken Language Understanding Modelscited by 24
  7. 2019Learning Alignment for Multimodal Emotion Recognition from Speechcited by 22
  8. 2020Pretrained Semantic Speech Embeddings for End-to-End Spoken Language Understanding via Cross-Modal Teacher-Student Learningcited by 14
  9. 2018Capsule Networks for Low Resource Spoken Language Understandingcited by 11
  10. 2019Leveraging Acoustic Cues and Paralinguistic Embeddings to Detect Expression from Voicecited by 7
  11. 2020Multitask Learning with Capsule Networks for Speech-to-Intent Applicationscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 1; 2023: 2; 2024: 1; 2025: 0; 2026: 1.

11 papers.

  1. 2021Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classificationcited by 14
  2. 2021Detecting Dementia from Speech and Transcripts using Transformerscited by 13
  3. 2021Timers and Such: A Practical Benchmark for Spoken Language Understanding with Numberscited by 11
  4. 2021Pre-training for Spoken Language Understanding with Joint Textual and Phonetic Representation Learningcited by 6
  5. 2023Leveraging Pretrained ASR Encoders for Effective and Efficient End-to-End Speech Intent Classification and Slot Fillingcited by 3
  6. 2024AccentFold: A Journey through African Accents for Zero-Shot ASR Adaptation to Target Accentscited by 3
  7. 2022Effectiveness of Text, Acoustic, and Lattice-based representations in Spoken Language Understanding taskscited by 2
  8. 2026Selective Capability Unlearning in End-to-End Spoken Language Understandingcited by 1
  9. 2021Exploring Teacher-Student Learning Approach for Multi-lingual Speech-to-Intent Classificationcited by 0
  10. 2021Oracle Teacher: Leveraging Target Information for Better Knowledge Distillation of CTC Modelscited by 0
  11. 2023Improving End-to-End Speech Processing by Efficient Text Data Utilization with Latent Synthesiscited by 0