mlsanityData to 7 Oct 2026

Semantic enrichment towards efficient speech representations

arXiv 2307.01323 ↗First posted 3 Jul 2023cs.CL cs.SD eess.AScited by 2

Not on the map. Why?

Abstract (arXiv)

Over the past few years, self-supervised learned speech representations have emerged as fruitful replacements for conventional surface representations when solving Spoken Language Understanding (SLU) tasks. Simultaneously, multilingual models trained on massive textual data were introduced to encode language agnostic semantics. Recently, the SAMU-XLSR approach introduced a way to make profit from such textual models to enrich multilingual speech representations with language agnostic semantics. By aiming for better semantic extraction on a challenging Spoken Language Understanding task and in consideration with computation costs, this study investigates a specific in-domain semantic enrichment of the SAMU-XLSR model by specializing it on a small amount of transcribed data from the downstream task. In addition, we show the benefits of the use of same-domain French and Italian benchmarks for low-resource language portability and explore cross-domain capacities of the enriched SAMU-XLSR.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 3 Jul 2023.

Builds on

20 of 29 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  5. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  6. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  7. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 196
  8. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 186
  9. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 158
  10. 2018Towards end-to-end spoken language understandingcited by 92
  11. 2021Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verificationcited by 71
  12. 2018From Audio to Semantics: Approaches to end-to-end spoken language understandingcited by 63
8 more
  1. 2020Leveraging Unpaired Text Data for Training End-to-End Speech-to-Intent Systemscited by 38
  2. 2022SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representationcited by 32
  3. 2019Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portabilitycited by 29
  4. 2022Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?cited by 27
  5. 2019Using Speech Synthesis to Train End-to-End Spoken Language Understanding Modelscited by 24
  6. 2020Tie Your Embeddings Down: Cross-Modal Latent Spaces for End-to-end Spoken Language Understandingcited by 23
  7. 2020On the use of Self-supervised Pre-trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognitioncited by 19
  8. 2022On the Use of Semantically-Aligned Speech Representations for Spoken Language Understandingcited by 7

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024New Semantic Task for the French Spoken Language Understanding MEDIA Benchmarkcited by 2
  2. 2024A dual task learning approach to fine-tune a multilingual semantic speech encoder for Spoken Language Understandingcited by 1