The X-LANCE Technical Report for Interspeech 2024 Speech Processing Using Discrete Speech Unit Challenge
Not on the map. Why?
Abstract (arXiv)
Discrete speech tokens have been more and more popular in multiple speech processing fields, including automatic speech recognition (ASR), text-to-speech (TTS) and singing voice synthesis (SVS). In this paper, we describe the systems developed by the SJTU X-LANCE group for the TTS (acoustic + vocoder), SVS, and ASR tracks in the Interspeech 2024 Speech Processing Using Discrete Speech Unit Challenge. Notably, we achieved 1st rank on the leaderboard in the TTS track both with the whole training set and only 1h training data, with the highest UTMOS score and lowest bitrate among all submissions.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 28 of its 35 references that are in the map. First posted on arXiv 9 Apr 2024.
Builds on
28 of 35 references.
References · 28
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
- 2022High Fidelity Neural Audio Compressioncited by 673
- 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
- 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
16 more
- 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
- 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
- 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
- 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervisioncited by 167
- 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 77
- 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 70
- 2022Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesiscited by 53
- 2024ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reorderingcited by 50
- 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
- 2023FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codeccited by 44
- 2023UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocodingcited by 37
- 2022VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Featurecited by 32
- 2023Make-A-Voice: Unified Voice Synthesis With Discrete Representationcited by 27
- 2023Towards Universal Speech Discrete Tokens: A Case Study for ASR and TTScited by 24
- 2023Vec-Tok Speech: speech vectorization and tokenization for neural speech generationcited by 12
Built on by
Citing papers per year: 2024: 1.
1 paper.