mlsanityData to 7 Oct 2026

OLISIA: a Cascade System for Spoken Dialogue State Tracking

arXiv 2304.11073 ↗First posted 20 Apr 2023eess.AS cs.AI cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Though Dialogue State Tracking (DST) is a core component of spoken dialogue systems, recent work on this task mostly deals with chat corpora, disregarding the discrepancies between spoken and written language.In this paper, we propose OLISIA, a cascade system which integrates an Automatic Speech Recognition (ASR) model and a DST model. We introduce several adaptations in the ASR and DST modules to improve integration and robustness to spoken conversations.With these adaptations, our system ranked first in DSTC11 Track 3, a benchmark to evaluate spoken DST. We conduct an in-depth analysis of the results and find that normalizing the ASR outputs and adapting the DST inputs through data augmentation, along with increasing the pre-trained models size all play an important role in reducing the performance discrepancy between written and spoken conversations.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 26 of its 41 references that are in the map. First posted on arXiv 20 Apr 2023.

Builds on

26 of 41 references.

References · 26

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  5. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  6. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 230
  7. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 177
  8. 2018Towards end-to-end spoken language understandingcited by 92
  9. 2020Few-shot Natural Language Generation for Task-Oriented Dialogcited by 89
  10. 2019Speech Recognition with Augmented Synthesized Speechcited by 60
  11. 2019Correction of Automatic Speech Recognition with Transformer Sequence-to-sequence Modelcited by 52
  12. 2020Using Synthetic Audio to Improve The Recognition of Out-Of-Vocabulary Words in End-To-End ASR Systemscited by 41
14 more
  1. 2020Robustness Testing of Language Understanding in Task-Oriented Dialogcited by 32
  2. 2021Annotation Inconsistency and Entity Bias in MultiWOZcited by 30
  3. 2021"How Robust r u?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversationscited by 30
  4. 2020You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentationcited by 29
  5. 2018Training Neural Speech Recognition Systems with Synthetic Speech Augmentationcited by 21
  6. 2020Data Augmentation for Training Dialog Models Robust to Speech Recognition Errorscited by 17
  7. 2021Revisiting the Boundary between ASR and NLU in the Age of Conversational Dialog Systemscited by 15
  8. 2020Improving Spoken Language Understanding By Exploiting ASR N-best Hypothesescited by 12
  9. 2021N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypothesescited by 12
  10. 2021TOD-DA: Towards Boosting the Robustness of Task-oriented Dialogue Modeling on Spoken Conversationscited by 11
  11. 2022End-to-end model for named entity recognition from speech without paired training datacited by 11
  12. 2020Dialogue history integration into end-to-end signal-to-concept spoken language understanding systemscited by 10
  13. 2016Exploiting Sentence and Context Representations in Deep Neural Models for Spoken Language Understandingcited by 3
  14. 2020Modeling ASR Ambiguity for Dialogue State Tracking Using Word Confusion Networkscited by 2

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 2.

5 papers.

  1. 2024Speech Retrieval-Augmented Generation without Automatic Speech Recognitioncited by 14
  2. 2025SEAL: Speech Embedding Alignment Learning for Speech Large Language Model with Retrieval-Augmented Generationcited by 3
  3. 2023Is one brick enough to break the wall of spoken dialogue state tracking?cited by 1
  4. 2025The Speech-LLM Takes It All: A Truly Fully End-to-End Spoken Dialogue State Tracking Approachcited by 1
  5. 2023Are cascade dialogue state tracking models speaking out of turn in spoken dialogues?cited by 0