mlsanityData to 14 Oct 2026

FlowDec: A flow-based full-band general audio codec with high perceptual quality

arXiv 2503.01485 ↗First posted 3 Mar 2025cs.SD cs.LG eess.AScited by 15

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

We propose FlowDec, a neural full-band audio codec for general audio sampled at 48 kHz that combines non-adversarial codec training with a stochastic postfilter based on a novel conditional flow matching method. Compared to the prior work ScoreDec which is based on score matching, we generalize from speech to general audio and move from 24 kbit/s to as low as 4 kbit/s, while improving output quality and reducing the required postfilter DNN evaluations from 60 to 6 without any fine-tuning or distillation techniques. We provide theoretical insights and geometric intuitions for our approach in comparison to ScoreDec as well as another recent work that uses flow matching, and conduct ablation studies on our proposed components. We show that FlowDec is a competitive alternative to the recent GAN-dominated stream of neural codecs, achieving FAD scores better than those of the established GAN-based codec DAC and listening test scores that are on par, and producing qualitatively more natural reconstructions for speech and harmonic structures in music.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 52 of its 78 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

52 of 78 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform Generation14 Aug
  2. 2024FlowAVSE: Efficient Audio-Visual Speech Enhancement with Conditional Flow Matching13 Jun
  3. 2023Reducing the Prior Mismatch of Stochastic Differential Equations for Diffusion-based Speech Enhancement28 Feb
  4. 2022StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation22 Dec
  5. 2022Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration4 Nov
  6. 2022Solving Audio Inverse Problems with a Diffusion Model27 Oct
  7. 2022Speech Enhancement and Dereverberation with Diffusion-based Generative Models11 Aug
  8. 2022Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain31 Mar
  9. 2021PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior11 Jun

Other references · 43

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2022Generative Adversarial Networkscited by 5,627
  7. 2017Neural Discrete Representation Learningcited by 4,712
  8. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  9. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  10. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  11. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 879
  12. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
31 more
  1. 2022High Fidelity Neural Audio Compressioncited by 687
  2. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  3. 2017The Perception-Distortion Tradeoffcited by 587
  4. 2023MusicLM: Generating Music From Textcited by 448
  5. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  6. 2018SDR - half-baked or well done?cited by 423
  7. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 354
  8. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 340
  9. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 304
  10. 2023Multisample Flow Matching: Straightening Flows with Minibatch Couplingscited by 278
  11. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 267
  12. 2018Efficient Neural Audio Synthesiscited by 219
  13. 2019Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoffcited by 211
  14. 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 110
  15. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 98
  16. 2023Adapting Frechet Audio Distance for Generative Music Evaluationcited by 92
  17. 2024EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberationcited by 64
  18. 2023AudioDec: An Open-source Streaming High-fidelity Neural Audio Codeccited by 47
  19. 2017Wavenet based low rate speech codingcited by 33
  20. 2017End-to-End Optimized Speech Coding with Deep Neural Networkscited by 22
  21. 2023From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusioncited by 22
  22. 2019A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNetcited by 16
  23. 2024ICASSP 2024 Speech Signal Improvement Challengecited by 16
  24. 2022DriftRec: Adapting diffusion models to blind JPEG restorationcited by 15
  25. 2018High-quality speech coding with SampleRNNcited by 14
  26. 2021Objective Measures of Perceptual Audio Quality Reviewed: An Evaluation of Their Application Domain Dependencecited by 14
  27. 2016High-Quality, Low-Delay Music Coding in the Opus Codeccited by 13
  28. 2020Audio Codec Enhancement with Generative Adversarial Networkscited by 12
  29. 2024ScoreDec: A Phase-preserving High-Fidelity Audio Codec with A Generalized Score-based Diffusion Post-filtercited by 7
  30. 2021A Streamwise GAN Vocoder for Wideband Speech Coding at Very Low Bit Ratecited by 5
  31. 2022PostGAN: A GAN-Based Post-Processor to Enhance the Quality of Coded Speechcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 8.

15 papers.

Sort:

By area:

  1. 2025Audio Super-Resolution with Latent Bridge Modelscited by 14
  2. 2026MOSS-TTS Technical Reportcited by 14
  3. 2025TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modelingcited by 13
  4. 2025BinauralFlow: A Causal and Streamable Approach for High-Quality Binaural Speech Synthesis with Flow Matching Modelscited by 5
  5. 2025SpectroStream: A Versatile Neural Codec for General Audiocited by 4
  6. 2025Real-Time Streamable Generative Speech Restoration with Flow Matchingcited by 4
  7. 2026Scaling Speech Tokenizers with Diffusion Autoencoderscited by 2
  8. 2025Real-Time Streaming Mel Vocoding with Generative Flow Matchingcited by 1
  9. 2025PitchFlower: A flow-based neural audio codec with pitch controllabilitycited by 1
  10. 2026EuleroDec: A Complex-Valued RVQ-VAE for Efficient and Robust Audio Codingcited by 1
  11. 2026Conditional Flow Matching for Visually-Guided Acoustic Highlightingcited by 1
  12. 2026A Fast Solver for Interpolating Stochastic Differential Equation Diffusion Models for Speech Restorationcited by 0
3 more
  1. 2026ReGen: Hierarchical Multi-Prompt Representation Generation for Efficient Waveform Diffusion Modelscited by 0
  2. 2026Rethinking Language Model-Based Generative Speech Enhancement in the Latent Space of a Neural Audio Codeccited by 0
  3. 2026Geometric Iterative Retrieval for Neural Audio Codec Resynthesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024FlowMAC: Conditional Flow Matching for Audio Coding at Low Bit Rates
  2. 2026CodecFlow: Efficient Bandwidth Extension via Conditional Flow Matching in Neural Codec Latent Space
  3. 2024Gull: A Generative Multifunctional Audio Codec
  4. 2025Flow2GAN: Hybrid Flow Matching and GAN with Multi-Resolution Network for Few-step High-Fidelity Audio Generation
  5. 2026UniSRCodec: Unified and Low-Bitrate Single Codebook Codec with Sub-Band Reconstruction
  6. 2025CodecSep: Prompt-Driven Universal Sound Separation on Neural Audio Codec Latents
  7. 2026GS-Codec: A Gaussian-Splatting Bottleneck for Neural Audio Coding
  8. 2022NESC: Robust Neural End-2-End Speech Coding with GANs