# Adapting Text-based Dialogue State Tracker for Spoken Dialogues

arXiv: https://arxiv.org/abs/2308.15053 · HTML: https://mlsanity.com/paper/2308.15053/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 29 Aug 2023.

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2](https://mlsanity.com/paper/2012.03539/) (2020)
- [Efficient Dialogue State Tracking by Selectively Overwriting Memory](https://mlsanity.com/paper/1911.03906/) (2019)
- [Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems](https://mlsanity.com/paper/1804.08217/) (2018)
- [Description-Driven Task-Oriented Dialog Modeling](https://mlsanity.com/paper/2201.08904/) (2022)
- [Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2008.06239/) (2020)
- [Robust Zero-Shot Cross-Domain Slot Filling with Example Values](https://mlsanity.com/paper/1906.06870/) (2019)
- ["How Robust r u?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversations](https://mlsanity.com/paper/2109.13489/) (2021)
- [Scalable and Accurate Dialogue State Tracking via Hierarchical Sequence Generation](https://mlsanity.com/paper/1909.00754/) (2019)
- [Error Correction in ASR using Sequence-to-Sequence Models](https://mlsanity.com/paper/2202.01157/) (2022)
- [TOD-DA: Towards Boosting the Robustness of Task-oriented Dialogue Modeling on Spoken Conversations](https://mlsanity.com/paper/2112.12441/) (2021)

## Built on by (most cited first, 2 of 2)

- [Speak & Spell: LLM-Driven Controllable Phonetic Error Augmentation for Robust Dialogue State Tracking](https://mlsanity.com/paper/2409.06263/) (2024)
- [Joint Speech and Text Training for LLM-Based End-to-End Spoken Dialogue State Tracking](https://mlsanity.com/paper/2511.22503/) (2025)

## Abstract (arXiv)

Although there have been remarkable advances in dialogue systems through the dialogue systems technology competition (DSTC), it remains one of the key challenges to building a robust task-oriented dialogue system with a speech interface. Most of the progress has been made for text-based dialogue systems since there are abundant datasets with written corpora while those with spoken dialogues are very scarce. However, as can be seen from voice assistant systems such as Siri and Alexa, it is of practical importance to transfer the success to spoken dialogues. In this paper, we describe our engineering effort in building a highly successful model that participated in the speech-aware dialogue systems technology challenge track in DSTC11. Our model consists of three major modules: (1) automatic speech recognition error correction to bridge the gap between the spoken and the text utterances, (2) text-based dialogue system (D3ST) for estimating the slots and values using slot descriptions, and (3) post-processing for recovering the error of the estimated slot value. Our experiments show that it is important to use an explicit automatic speech recognition error correction module, post-processing, and data augmentation to adapt a text-based dialogue state tracker for spoken dialogue corpora.