mlsanityData to 14 Oct 2026

Improving End-to-End Models for Set Prediction in Spoken Language Understanding

arXiv 2201.12105 ↗First posted 28 Jan 2022cs.CL cs.LG cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

The goal of spoken language understanding (SLU) systems is to determine the meaning of the input speech signal, unlike speech recognition which aims to produce verbatim transcripts. Advances in end-to-end (E2E) speech modeling have made it possible to train solely on semantic entities, which are far cheaper to collect than verbatim transcripts. We focus on this set prediction problem, where entity order is unspecified. Using two classes of E2E models, RNN transducers and attention based encoder-decoders, we show that these models work best when the training entity sequence is arranged in spoken order. To improve E2E SLU models when entity spoken order is unknown, we propose a novel data augmentation technique along with an implicit attention based alignment method to infer the spoken order. F1 scores significantly increased by more than 11% for RNN-T and about 2% for attention based encoder-decoder SLU models, outperforming previously reported results.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 28 Jan 2022.

Builds on

26 of 35 references.

References · 26

  1. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2015Attention-Based Models for Speech Recognitioncited by 401
  4. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 234
  5. 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 193
  6. 2018Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducercited by 96
  7. 2019Improving RNN Transducer Modeling for End-to-End Speech Recognitioncited by 68
  8. 2018From Audio to Semantics: Approaches to end-to-end spoken language understandingcited by 63
  9. 2016Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activationscited by 54
  10. 2019Joint Speech Recognition and Speaker Diarization via Sequence Transductioncited by 49
  11. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  12. 2020Leveraging Unpaired Text Data for Training End-to-End Speech-to-Intent Systemscited by 38
14 more
  1. 2020Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboardcited by 36
  2. 2020End-to-End Neural Transformer Based Spoken Language Understandingcited by 31
  3. 2021On the limit of English conversational speech recognitioncited by 30
  4. 2019Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portabilitycited by 29
  5. 2020Tie Your Embeddings Down: Cross-Modal Latent Spaces for End-to-end Spoken Language Understandingcited by 23
  6. 2016Joint Online Spoken Language Understanding and Language Modeling with Recurrent Neural Networkscited by 19
  7. 2020End-to-End Spoken Language Understanding Without Full Transcriptscited by 19
  8. 2019Guiding CTC Posterior Spike Timings for Improved Posterior Fusion and Knowledge Distillationcited by 18
  9. 2020Exploring Transfer Learning For End-to-End Spoken Language Understandingcited by 17
  10. 2020Adapting Pretrained Transformer to Lattices for Spoken Language Understandingcited by 12
  11. 2019End-to-end architectures for ASR-free spoken language understandingcited by 11
  12. 2020End-to-end spoken language understanding using transformer networks and self-supervised pre-trained featurescited by 8
  13. 2021RNN Transducer Models For Spoken Language Understandingcited by 7
  14. 2020Style Attuned Pre-training and Parameter Efficient Fine-tuning for Spoken Language Understandingcited by 2

Built on by

Not cited yet.