mlsanityData to 7 Oct 2026

Toward Low-Cost End-to-End Spoken Language Understanding

arXiv 2207.00352 ↗First posted 1 Jul 2022cs.CL cs.SD eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

Recent advances in spoken language understanding benefited from Self-Supervised models trained on large speech corpora. For French, the LeBenchmark project has made such models available and has led to impressive progress on several tasks including spoken language understanding. These advances have a non-negligible cost in terms of computation time and energy consumption. In this paper, we compare several learning strategies trying to reduce such cost while keeping competitive performance. At the same time we propose an extensive analysis where we measure the cost of our models in terms of training time and electric energy consumption, hopefully promoting a comprehensive evaluation procedure. The experiments are performed on the FSC and MEDIA corpora, and show that it is possible to reduce the learning cost while maintaining state-of-the-art performance and using SSL models.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 20 of its 43 references that are in the map. First posted on arXiv 1 Jul 2022.

Builds on

20 of 43 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  6. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  7. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  8. 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 191
  9. 2018Towards end-to-end spoken language understandingcited by 92
  10. 2021LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speechcited by 46
  11. 2020Semi-Supervised Spoken Language Understanding via Self-Supervised Speech and Language Model Pretrainingcited by 35
  12. 2020End-to-End Neural Transformer Based Spoken Language Understandingcited by 31
8 more
  1. 2019Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portabilitycited by 29
  2. 2021Speech-language Pre-training for End-to-end Spoken Language Understandingcited by 27
  3. 2019End-to-end architectures for ASR-free spoken language understandingcited by 11
  4. 2021Where are we in semantic concept extraction for Spoken Language Understanding?cited by 11
  5. 2021End2End Acoustic to Semantic Transductioncited by 10
  6. 2020A Data Efficient End-To-End Spoken Language Understanding Architecturecited by 8
  7. 2016Improving Recurrent Neural Networks For Sequence Labellingcited by 3
  8. 2017Label-Dependencies Aware Recurrent Neural Networkscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 0; 2026: 1.

3 papers.

  1. 2023LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speechcited by 17
  2. 2022Bridging Speech and Textual Pre-trained Models with Unsupervised ASRcited by 3
  3. 2026Assessing the Energy and Carbon Emissions of Neural Speaker Verification Model in Training and Inferencecited by 0