mlsanityData to 7 Oct 2026

Integrating Dialog History into End-to-End Spoken Language Understanding Systems

arXiv 2108.08405 ↗First posted 18 Aug 2021cs.CL cs.SD eess.AScited by 9

Not on the map. Why?

Abstract (arXiv)

End-to-end spoken language understanding (SLU) systems that process human-human or human-computer interactions are often context independent and process each turn of a conversation independently. Spoken conversations on the other hand, are very much context dependent, and dialog history contains useful information that can improve the processing of each conversational turn. In this paper, we investigate the importance of dialog history and how it can be effectively integrated into end-to-end SLU systems. While processing a spoken utterance, our proposed RNN transducer (RNN-T) based SLU model has access to its dialog history in the form of decoded transcripts and SLU labels of previous turns. We encode the dialog history as BERT embeddings, and use them as an additional input to the SLU model along with the speech features for the current utterance. We evaluate our approach on a recently released spoken dialog data set, the HarperValleyBank corpus. We observe significant improvements: 8% for dialog action and 30% for caller intent recognition tasks, in comparison to a competitive context independent end-to-end baseline system.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 26 of its 41 references that are in the map. First posted on arXiv 18 Aug 2021.

Builds on

26 of 41 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
  3. 2017Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Ratescited by 196
  4. 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 191
  5. 2018Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducercited by 95
  6. 2018Towards end-to-end spoken language understandingcited by 92
  7. 2020Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbotscited by 73
  8. 2019Improving RNN Transducer Modeling for End-to-End Speech Recognitioncited by 68
  9. 2018From Audio to Semantics: Approaches to end-to-end spoken language understandingcited by 63
  10. 2016Tracking the World State with Recurrent Entity Networkscited by 51
  11. 2019Joint Speech Recognition and Speaker Diarization via Sequence Transductioncited by 49
  12. 2019Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Studycited by 47
14 more
  1. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  2. 2020Leveraging Unpaired Text Data for Training End-to-End Speech-to-Intent Systemscited by 38
  3. 2020End-to-End Neural Transformer Based Spoken Language Understandingcited by 31
  4. 2019Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portabilitycited by 29
  5. 2020End-to-End Spoken Language Understanding Without Full Transcriptscited by 19
  6. 2020Improving End-to-End Speech-to-Intent Classification with Reptilecited by 17
  7. 2019Guiding CTC Posterior Spike Timings for Improved Posterior Fusion and Knowledge Distillationcited by 16
  8. 2019HyST: A Hybrid Approach for Flexible and Accurate Dialogue State Trackingcited by 14
  9. 2019End-to-end architectures for ASR-free spoken language understandingcited by 11
  10. 2020Large-scale Transfer Learning for Low-resource Spoken Language Understandingcited by 11
  11. 2020Dialogue history integration into end-to-end signal-to-concept spoken language understanding systemscited by 10
  12. 2020HarperValleyBank: A Domain-Specific Spoken Dialog Corpuscited by 10
  13. 2021RNN Transducer Models For Spoken Language Understandingcited by 7
  14. 2018An Efficient Approach to Encoding Context for Spoken Language Understandingcited by 4

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 5; 2023: 2; 2024: 0; 2025: 1.

9 papers.

  1. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  2. 2022Integrating Text Inputs For Training and Adapting RNN Transducer ASR Modelscited by 20
  3. 2022Towards End-to-End Integration of Dialog History for Improved Spoken Language Understandingcited by 8
  4. 2022Two-Pass Low Latency End-to-End Spoken Language Understandingcited by 8
  5. 2022Context-aware Fine-tuning of Self-supervised Speech Modelscited by 7
  6. 2023Joint Modelling of Spoken Language Understanding Tasks with Integrated Dialog Historycited by 3
  7. 2023Is one brick enough to break the wall of spoken dialogue state tracking?cited by 1
  8. 2025Joint Speech and Text Training for LLM-Based End-to-End Spoken Dialogue State Trackingcited by 1
  9. 2022Meta Auxiliary Learning for Low-resource Spoken Language Understandingcited by 0