mlsanityData to 14 Oct 2026

On the Use of Semantically-Aligned Speech Representations for Spoken Language Understanding

arXiv 2210.05291 ↗First posted 11 Oct 2022cs.CL cs.SD eess.AScited by 7

Not on the map. Why?

Abstract (arXiv)

In this paper we examine the use of semantically-aligned speech representations for end-to-end spoken language understanding (SLU). We employ the recently-introduced SAMU-XLSR model, which is designed to generate a single embedding that captures the semantics at the utterance level, semantically aligned across different languages. This model combines the acoustic frame-level speech representation learning model (XLS-R) with the Language Agnostic BERT Sentence Embedding (LaBSE) model. We show that the use of the SAMU-XLSR model instead of the initial XLS-R model improves significantly the performance in the framework of end-to-end SLU. Finally, we present the benefits of using this model towards language portability in SLU.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 11 Oct 2022.

Builds on

24 of 38 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  6. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  7. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 198
  8. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 188
  9. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 162
  10. 2018Towards end-to-end spoken language understandingcited by 92
  11. 2021Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verificationcited by 78
  12. 2018From Audio to Semantics: Approaches to end-to-end spoken language understandingcited by 63
12 more
  1. 2020Leveraging Unpaired Text Data for Training End-to-End Speech-to-Intent Systemscited by 38
  2. 2022SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representationcited by 32
  3. 2020End-to-End Neural Transformer Based Spoken Language Understandingcited by 31
  4. 2019Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portabilitycited by 29
  5. 2022Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?cited by 27
  6. 2019Using Speech Synthesis to Train End-to-End Spoken Language Understanding Modelscited by 24
  7. 2020Tie Your Embeddings Down: Cross-Modal Latent Spaces for End-to-end Spoken Language Understandingcited by 23
  8. 2020On the use of Self-supervised Pre-trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognitioncited by 19
  9. 2019End-to-end architectures for ASR-free spoken language understandingcited by 11
  10. 2021Where are we in semantic concept extraction for Spoken Language Understanding?cited by 11
  11. 2020A Data Efficient End-To-End Spoken Language Understanding Architecturecited by 8
  12. 2022Measuring the Impact of Individual Domain Factors in Self-Supervised Pre-Trainingcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 1; 2025: 1.

7 papers.

  1. 2023LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speechcited by 18
  2. 2023LibriSQA: A Novel Dataset and Framework for Spoken Question Answering with Large Language Modelscited by 14
  3. 2025SENSE models: an open source solution for multilingual and multimodal semantic-based taskscited by 6
  4. 2023Improved Cross-Lingual Transfer Learning For Automatic Speech Translationcited by 2
  5. 2023Semantic enrichment towards efficient speech representationscited by 2
  6. 2023Leveraging Multilingual Self-Supervised Pretrained Models for Sequence-to-Sequence End-to-End Spoken Language Understandingcited by 1
  7. 2024A dual task learning approach to fine-tune a multilingual semantic speech encoder for Spoken Language Understandingcited by 1