# MuCodec: Ultra Low-Bitrate Music Codec

arXiv: https://arxiv.org/abs/2409.13216 · HTML: https://mlsanity.com/paper/2409.13216/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2024.

## Built on by (most cited first, 9 of 9)

- [WavChat: A Survey of Spoken Dialogue Models](https://mlsanity.com/paper/2411.13577/) (2024)
- [LeVo: High-Quality Song Generation with Multi-Preference Alignment](https://mlsanity.com/paper/2506.07520/) (2025)
- [SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement](https://mlsanity.com/paper/2506.07634/) (2025)
- [Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control](https://mlsanity.com/paper/2601.03973/) (2026)
- [LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling and Progressive Post-Training](https://mlsanity.com/paper/2606.30642/) (2026)
- [On the Design of Diffusion-based Neural Speech Codecs](https://mlsanity.com/paper/2504.08470/) (2025)
- [DuoTok: Source-Aware Dual-Track Music Tokenization for Vocal-Accompaniment Generation](https://mlsanity.com/paper/2511.20224/) (2025)
- [S-PRESSO: Ultra Low Bitrate Sound Effect Compression With Diffusion Autoencoders And Offline Quantization](https://mlsanity.com/paper/2602.15082/) (2026)
- [SqueezeComposer: Temporal Speed-up is A Simple Trick for Long-form Music Composing](https://mlsanity.com/paper/2603.21073/) (2026)

## Abstract (arXiv)

Music codecs are a vital aspect of audio codec research, and ultra low-bitrate compression holds significant importance for music transmission and generation. Due to the complexity of music backgrounds and the richness of vocals, solely relying on modeling semantic or acoustic information cannot effectively reconstruct music with both vocals and backgrounds. To address this issue, we propose MuCodec, specifically targeting music compression and reconstruction tasks at ultra low bitrates. MuCodec employs MuEncoder to extract both acoustic and semantic features, discretizes them with RVQ, and obtains Mel-VAE features via flow-matching. The music is then reconstructed using a pre-trained MEL-VAE decoder and HiFi-GAN. MuCodec can reconstruct high-fidelity music at ultra low (0.35kbps) or high bitrates (1.35kbps), achieving the best results to date in both subjective and objective metrics. Code and Demo: https://xuyaoxun.github.io/MuCodec_demo/.