mlsanityData to 14 Oct 2026

Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target

arXiv 2305.18096 ↗First posted 29 May 2023cs.CL eess.AScited by 6

Not on the map. Why?

Abstract (arXiv)

Spoken Language Understanding (SLU) is a task that aims to extract semantic information from spoken utterances. Previous research has made progress in end-to-end SLU by using paired speech-text data, such as pre-trained Automatic Speech Recognition (ASR) models or paired text as intermediate targets. However, acquiring paired transcripts is expensive and impractical for unwritten languages. On the other hand, Textless SLU extracts semantic information from speech without utilizing paired transcripts. However, the absence of intermediate targets and training guidance for textless SLU often results in suboptimal performance. In this work, inspired by the content-disentangled discrete units from self-supervised speech models, we proposed to use discrete units as intermediate guidance to improve textless SLU performance. Our method surpasses the baseline method on five SLU benchmark corpora. Additionally, we find that unit guidance facilitates few-shot learning and enhances the model's ability to handle noise.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 27 of its 29 references that are in the map. First posted on arXiv 29 May 2023.

Builds on

27 of 29 references.

References · 27

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  4. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  5. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
  6. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 377
  7. 2021Generative Spoken Language Modeling from Raw Audiocited by 340
  8. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  9. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  10. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 198
  11. 2021Unsupervised Speech Recognitioncited by 148
  12. 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
15 more
  1. 2018Towards end-to-end spoken language understandingcited by 92
  2. 2021SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speechcited by 72
  3. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  4. 2022On the Utility of Self-supervised Models for Prosody-related Taskscited by 42
  5. 2020Semi-Supervised Spoken Language Understanding via Self-Supervised Speech and Language Model Pretrainingcited by 35
  6. 2022textless-lib: a Library for Textless Spoken Language Processingcited by 28
  7. 2022STOP: A dataset for Spoken Task Oriented Semantic Parsingcited by 22
  8. 2022Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systemscited by 20
  9. 2020End-to-End Spoken Language Understanding Without Full Transcriptscited by 19
  10. 2021End-to-End Spoken Language Understanding for Generalized Voice Assistantscited by 17
  11. 2022DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoken Question Answeringcited by 15
  12. 2021Analyzing the Robustness of Unsupervised Speech Recognitioncited by 11
  13. 2020End-to-end spoken language understanding using transformer networks and self-supervised pre-trained featurescited by 8
  14. 2021RNN Transducer Models For Spoken Language Understandingcited by 7
  15. 2021Speech2Slot: An End-to-End Knowledge-based Slot Filling from Speechcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 4; 2025: 1.

6 papers.

  1. 2024Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedbackcited by 23
  2. 2024LAST: Language Model Aware Speech Tokenizationcited by 9
  3. 2024Self-Supervised Syllable Discovery Based on Speaker-Disentangled HuBERTcited by 4
  4. 2024Integrating Self-supervised Speech Model with Pseudo Word-level Targets from Visually-grounded Speech Modelcited by 1
  5. 2023HuBERTopic: Enhancing Semantic Representation of HuBERT through Self-supervision Utilizing Topic Modelcited by 0
  6. 2025Representing Speech Through Autoregressive Prediction of Cochlear Tokenscited by 0