mlsanityData to 7 Oct 2026

OLIVE: View-Augmented Latent Prediction with Waveform Reconstruction for Speech SSL

arXiv 2606.30356 ↗First posted 29 Jun 2026cs.CL cs.LG cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose Online Latent prediction with Invariant Views and rEconstruction (OLIVE), a self-supervised speech representation learning framework that jointly optimizes analysis and synthesis objectives. OLIVE combines view-augmented masked latent prediction with waveform reconstruction under a unified objective. Reconstruction constrains early encoder features to retain signal-level information, while masked latent prediction shapes later contextual representations toward invariance for robust downstream performance. We show that these objectives enable representations that support a broad range of tasks. In particular, OLIVE improves results on generation and speaker tasks, maintains competitive performance on recognition and semantic tasks, and improves waveform reconstruction.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 48 of its 65 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

48 of 65 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  9. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  10. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  11. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  12. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
36 more
  1. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  2. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  3. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  4. 2022High Fidelity Neural Audio Compressioncited by 673
  5. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  6. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  7. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  8. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  9. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
  10. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  11. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  12. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  13. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 196
  14. 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 185
  15. 2022Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Languagecited by 108
  16. 2021BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representationcited by 106
  17. 2019Generative Pre-Training for Speech with Autoregressive Predictive Codingcited by 82
  18. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  19. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  20. 2020DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantizationcited by 77
  21. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 76
  22. 2023Voice Conversion With Just Nearest Neighborscited by 67
  23. 2022Audio Self-supervised Learning: A Surveycited by 65
  24. 2020Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependenciescited by 47
  25. 2022SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learningcited by 26
  26. 2022WavThruVec: Latent speech representation as intermediate features for neural speech synthesiscited by 17
  27. 2025NanoCodec: Towards High-Quality Ultra Fast Speech LLM Inferencecited by 14
  28. 2025Metis: A Foundation Speech Generation Model with Masked Generative Pre-trainingcited by 12
  29. 2022Self-Supervised Learning for Speech Enhancement through Synthesiscited by 10
  30. 2022BYOL-S: Learning Self-supervised Speech Representations by Bootstrappingcited by 9
  31. 2025WavJEPA: Semantic learning unlocks robust audio foundation models for raw waveformscited by 7
  32. 2024vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoderscited by 6
  33. 2025UniWav: Towards Unified Pre-training for Speech Representation Learning and Generationcited by 5
  34. 2024kNN Retrieval for Simple and Effective Zero-Shot Multi-speaker Text-to-Speechcited by 3
  35. 2024A Closer Look at Neural Codec Resynthesis: Bridging the Gap between Codec and Waveform Generationcited by 3
  36. 2025JEPA as a Neural Tokenizer: Learning Robust Speech Representations with Density Adaptive Attentioncited by 3

Built on by

Not cited yet.