# FlowMAC: Conditional Flow Matching for Audio Coding at Low Bit Rates

arXiv: https://arxiv.org/abs/2409.17635 · HTML: https://mlsanity.com/paper/2409.17635/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 31 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 25 of 25)

- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech](https://mlsanity.com/paper/2105.06337/) (2021)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale](https://mlsanity.com/paper/2306.15687/) (2023)
- [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://mlsanity.com/paper/2206.04658/) (2022)
- [Matcha-TTS: A fast TTS architecture with conditional flow matching](https://mlsanity.com/paper/2309.03199/) (2023)
- [Neural Networks Fail to Learn Periodic Functions and How to Fix It](https://mlsanity.com/paper/2006.08195/) (2020)
- [SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound](https://mlsanity.com/paper/2405.00233/) (2024)
- [FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codec](https://mlsanity.com/paper/2309.07405/) (2023)
- [Wavenet based low rate speech coding](https://mlsanity.com/paper/1712.01120/) (2017)
- [End-to-End Optimized Speech Coding with Deep Neural Networks](https://mlsanity.com/paper/1710.09064/) (2017)
- [From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion](https://mlsanity.com/paper/2308.02560/) (2023)
- [A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet](https://mlsanity.com/paper/1903.12087/) (2019)
- [Cascaded Cross-Module Residual Learning towards Lightweight End-to-End Speech Coding](https://mlsanity.com/paper/1906.07769/) (2019)
- [Generative De-Quantization for Neural Speech Codec via Latent Diffusion](https://mlsanity.com/paper/2311.08330/) (2023)
- [Efficient And Scalable Neural Residual Waveform Coding With Collaborative Quantization](https://mlsanity.com/paper/2002.05604/) (2020)
- [End-to-End Neural Speech Coding for Real-Time Communications](https://mlsanity.com/paper/2201.09429/) (2022)
- [A Streamwise GAN Vocoder for Wideband Speech Coding at Very Low Bit Rate](https://mlsanity.com/paper/2108.04051/) (2021)
- [NESC: Robust Neural End-2-End Speech Coding with GANs](https://mlsanity.com/paper/2207.03282/) (2022)

## Built on by (most cited first, 3 of 3)

- [FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks](https://mlsanity.com/paper/2502.04465/) (2025)
- [PitchFlower: A flow-based neural audio codec with pitch controllability](https://mlsanity.com/paper/2510.25566/) (2025)
- [Adapting Diffusion-Based Music Synthesis to Speech and Singing Voice Conversion](https://mlsanity.com/paper/2607.13278/) (2026)

## Abstract (arXiv)

This paper introduces FlowMAC, a novel neural audio codec for high-quality general audio compression at low bit rates based on conditional flow matching (CFM). FlowMAC jointly learns a mel spectrogram encoder, quantizer and decoder. At inference time the decoder integrates a continuous normalizing flow via an ODE solver to generate a high-quality mel spectrogram. This is the first time that a CFM-based approach is applied to general audio coding, enabling a scalable, simple and memory efficient training. Our subjective evaluations show that FlowMAC at 3 kbps achieves similar quality as state-of-the-art GAN-based and DDPM-based neural audio codecs at double the bit rate. Moreover, FlowMAC offers a tunable inference pipeline, which permits to trade off complexity and quality. This enables real-time coding on CPU, while maintaining high perceptual quality.