Exploring the Viability of Synthetic Audio Data for Audio-Based Dialogue State Tracking
Not on the map. Why?
Abstract (arXiv)
Dialogue state tracking plays a crucial role in extracting information in task-oriented dialogue systems. However, preceding research are limited to textual modalities, primarily due to the shortage of authentic human audio datasets. We address this by investigating synthetic audio data for audio-based DST. To this end, we develop cascading and end-to-end models, train them with our synthetic audio dataset, and test them on actual human speech data. To facilitate evaluation tailored to audio modalities, we introduce a novel PhonemeF1 to capture pronunciation similarity. Experimental results showed that models trained solely on synthetic datasets can generalize their performance to human voice data. By eliminating the dependency on human speech data collection, these insights pave the way for significant practical advancements in audio-based DST. Data and code are available at https://github.com/JihyunLee1/E2E-DST.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 4 Dec 2023.
Builds on
22 of 34 references.
References · 22
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
- 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 216
- 2021GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injectioncited by 102
- 2020Self-training and Pre-training are Complementary for Speech Recognitioncited by 99
- 2018Towards end-to-end spoken language understandingcited by 92
- 2020From Machine Reading Comprehension to Dialogue State Tracking: Bridging the Gapcited by 40
10 more
- 2019BERT-DST: Scalable End-to-End Dialogue State Tracking with Bidirectional Encoder Representations from Transformercited by 38
- 2020LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimizationcited by 30
- 2021Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognitioncited by 24
- 2022Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systemscited by 20
- 2020UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialoguescited by 19
- 2023Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systemscited by 19
- 2021EAT: Enhanced ASR-TTS for Self-supervised Speech Recognitioncited by 13
- 2020Replacing Human Audio with Synthetic Audio for On-device Unspoken Punctuation Predictioncited by 8
- 2022Computer-assisted Pronunciation Training -- Speech synthesis is almost all you needcited by 4
- 2022Audio-to-Intent Using Acoustic-Textual Subword Representations from End-to-End ASRcited by 4
Built on by
Citing papers per year: 2024: 1.
1 paper.