# CodecSlime: Temporal Redundancy Compression of Neural Speech Codec via Dynamic Frame Rate

arXiv: https://arxiv.org/abs/2506.21074 · HTML: https://mlsanity.com/paper/2506.21074/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2025.

## Built on by (most cited first, 4 of 4)

- [Beyond Fixed Frames: Dynamic Character-Aligned Speech Tokenization](https://mlsanity.com/paper/2601.23174/) (2026)
- [Do We Need Distinct Representations for Every Speech Token? Unveiling and Exploiting Redundancy in Large Speech Language Models](https://mlsanity.com/paper/2604.06871/) (2026)
- [Elastic Time: Dynamic Frame Rate Bottlenecks for Neural Audio Coding](https://mlsanity.com/paper/2606.27320/) (2026)
- [LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs](https://mlsanity.com/paper/2609.17509/) (2026)

## Abstract (arXiv)

Neural speech codecs have been widely used in audio compression and various downstream tasks. Current mainstream codecs are fixed-frame-rate (FFR), which allocate the same number of tokens to every equal-duration slice. However, speech is inherently non-uniform in temporal information density. As a result, many tokens are wasted on steady-state segments like long vowels and silences. To address this mismatch, we present CodecSlime, a plugin-style method for compressing temporal redundancy through supporting dynamic frame rate (DFR) on neural speech codecs for the first time. Our method is unsupervised and architecture-agnostic, combining two key innovations, ScheDFR and Melt-and-Cool, for adapting inference and training, respectively. When integrated into a typical VQ-GAN codec backbone and operating at 40 Hz DFR (≈ 600 bps), the reconstruction WER of CodecSlime is reduced by up to 32% relative to conventional FFR baselines with the same model architecture and similar bitrates, while other metrics are also competitive. CodecSlime also enables flexible trade-offs between reconstruction quality and bitrate: a single model supports inference at multiple frame rates and consistently outperforms FFR models at the corresponding frame rates. Audio samples are available at https://acadarmeria.github.io/codecslime/.