mlsanityData to 7 Oct 2026

LASER: Learning by Aligning Self-supervised Representations of Speech for Improving Content-related Tasks

arXiv 2406.09153 ↗First posted 13 Jun 2024cs.CL cs.SD eess.AScited by 2

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL)-based speech models are extensively used for full-stack speech processing. However, it has been observed that improving SSL-based speech representations using unlabeled speech for content-related tasks is challenging and computationally expensive. Recent attempts have been made to address this issue with cost-effective self-supervised fine-tuning (SSFT) approaches. Continuing in this direction, a cost-effective SSFT method named "LASER: Learning by Aligning Self-supervised Representations" is presented. LASER is based on the soft-DTW alignment loss with temporal regularisation term. Experiments are conducted with HuBERT and WavLM models and evaluated on the SUPERB benchmark for two content-related tasks: automatic speech recognition (ASR) and phoneme recognition (PR). A relative improvement of 3.7% and 8.2% for HuBERT, and 4.1% and 11.7% for WavLM are observed, for the ASR and PR tasks respectively, with only < 3 hours of fine-tuning on a single GPU.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 21 of its 27 references that are in the map. First posted on arXiv 13 Jun 2024.

Builds on

21 of 27 references.

References · 21

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  5. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  6. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
  7. 2017Soft-DTW: a Differentiable Loss Function for Time-Seriescited by 265
  8. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 186
  9. 2020Better Fine-Tuning by Reducing Representational Collapsecited by 156
  10. 2021DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERTcited by 129
  11. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  12. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 73
9 more
  1. 2021Learning by Aligning Videos in Timecited by 47
  2. 2018Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder modelscited by 29
  3. 2020Differentiable Divergences Between Time Seriescited by 26
  4. 2023Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clusteringcited by 21
  5. 2022Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptationcited by 16
  6. 2020DeepNAG: Deep Non-Adversarial Gesture Generationcited by 9
  7. 2023Soft Dynamic Time Warping for Multi-Pitch Estimation and Beyondcited by 4
  8. 2024SCORE: Self-supervised Correspondence Fine-tuning for Improved Content Representationscited by 3
  9. 2024Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representationscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025A Differentiable Alignment Framework for Sequence-to-Sequence Modeling via Optimal Transportcited by 0
  2. 2026Position-invariant Fine-tuning of Speech Enhancement Models with Self-supervised Speech Representationscited by 0