mlsanityData to 7 Oct 2026

RFWave: Multi-band Rectified Flow for Audio Waveform Reconstruction

arXiv 2403.05010 ↗First posted 8 Mar 2024cs.SD cs.AI eess.AScited by 9

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Recent advancements in generative modeling have significantly enhanced the reconstruction of audio waveforms from various representations. While diffusion models are adept at this task, they are hindered by latency issues due to their operation at the individual sample point level and the need for numerous sampling steps. In this study, we introduce RFWave, a cutting-edge multi-band Rectified Flow approach designed to reconstruct high-fidelity audio waveforms from Mel-spectrograms or discrete acoustic tokens. RFWave uniquely generates complex spectrograms and operates at the frame level, processing all subbands simultaneously to boost efficiency. Leveraging Rectified Flow, which targets a straight transport trajectory, RFWave achieves reconstruction with just 10 sampling steps. Our empirical evaluations show that RFWave not only provides outstanding reconstruction quality but also offers vastly superior computational efficiency, enabling audio generation at speeds up to 160 times faster than real-time on a GPU. An online demonstration is available at: https://rfwave-demo.github.io/rfwave/.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 56 of its 74 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

56 of 74 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Generative Pre-training for Speech with Flow Matching25 Oct
  2. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis21 Apr
  3. 2022SpecGrad: Diffusion Probabilistic Model based Neural Vocoder with Adaptive Noise Spectral Shaping31 Mar
  4. 2021PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior11 Jun
  5. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2022Flow Matching for Generative Modelingcited by 5,426
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  8. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  9. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  10. 2021Variational Diffusion Modelscited by 1,145
  11. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  12. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
38 more
  1. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  2. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  3. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  4. 2022High Fidelity Neural Audio Compressioncited by 673
  5. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  6. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  7. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  8. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  9. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 339
  10. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  11. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 245
  12. 2018Adversarial Audio Synthesiscited by 222
  13. 2018Efficient Neural Audio Synthesiscited by 218
  14. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  15. 2019Understanding and Improving Layer Normalizationcited by 213
  16. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 150
  17. 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
  18. 2020AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselinescited by 111
  19. 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 109
  20. 2023LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpuscited by 100
  21. 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 97
  22. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 93
  23. 2020ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metriccited by 76
  24. 2018ClariNet: Parallel Wave Generation in End-to-End Text-to-Speechcited by 70
  25. 2020Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speechcited by 63
  26. 2021Hi-Fi Multi-Speaker English TTS Datasetcited by 60
  27. 2022Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesiscited by 53
  28. 2021VoiceFixer: Toward General Speech Restoration with Neural Vocodercited by 50
  29. 2017A Wavenet for Speech Denoisingcited by 47
  30. 2021Chunked Autoregressive GAN for Conditional Waveform Synthesiscited by 38
  31. 2022iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transformcited by 30
  32. 2023From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusioncited by 22
  33. 2018Fast Spectrogram Inversion using Multi-head Convolutional Neural Networkscited by 21
  34. 2019JVS corpus: free Japanese multi-speaker voice corpuscited by 20
  35. 2018On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networkscited by 16
  36. 2023CML-TTS A Multilingual Dataset for Speech Synthesis in Low-Resource Languagescited by 15
  37. 2020JVS-MuSiC: Japanese multispeaker singing-voice corpuscited by 14
  38. 2024FreGrad: Lightweight and Fast Frequency-aware Diffusion Vocodercited by 11

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 2.

9 papers.

  1. 2025BinauralFlow: A Causal and Streamable Approach for High-Quality Binaural Speech Synthesis with Flow Matching Modelscited by 5
  2. 2025Flow Straight and Fast in Hilbert Space: Functional Rectified Flowcited by 5
  3. 2025Flow2GAN: Hybrid Flow Matching and GAN with Multi-Resolution Network for Few-step High-Fidelity Audio Generationcited by 4
  4. 2025DAFMSVC: One-Shot Singing Voice Conversion with Dual Attention Mechanism and Flow Matchingcited by 2
  5. 2025YingMusic-SVC: Real-World Robust Zero-Shot Singing Voice Conversion with Flow-GRPO and Singing-Specific Inductive Biasescited by 2
  6. 2025Spiking Vocos: An Energy-Efficient Neural Vocodercited by 0
  7. 2025BridgeVoC: Revitalizing Neural Vocoder from a Restoration Perspectivecited by 0
  8. 2026Toward Complex-Valued Neural Networks for Waveform Generationcited by 0
  9. 2026ReGen: Hierarchical Multi-Prompt Representation Generation for Efficient Waveform Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025FLowHigh: Towards Efficient and High-Quality Audio Super-Resolution with Single-Step Flow Matching
  2. 2026FastWave: Optimized Diffusion Model for Audio Super-Resolution
  3. 2023Fast Sampling generative model for Ultrasound image reconstruction
  4. 2023EDMSound: Spectrogram Based Diffusion Models for Efficient and High-Quality Audio Synthesis
  5. 2026WavFlow: Audio Generation in Waveform Space
  6. 2026Exploring Efficient Waveform Diffusion Models for Foley Sound Generation
  7. 2026SF-Flow: Sound field magnitude estimation via flow matching guided by sparse measurements
  8. 2024FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation