# Elastic Time: Dynamic Frame Rate Bottlenecks for Neural Audio Coding

arXiv: https://arxiv.org/abs/2606.27320 · HTML: https://mlsanity.com/paper/2606.27320/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 44 references that are in the map.
- First posted on arXiv 25 Jun 2026.

## Builds on (most cited first, 30 of 33)

- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [Muon is Scalable for LLM Training](https://mlsanity.com/paper/2502.16982/) (2025)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [AudioGen: Textually Guided Audio Generation](https://mlsanity.com/paper/2209.15352/) (2022)
- [Diffusion Transformers with Representation Autoencoders](https://mlsanity.com/paper/2510.11690/) (2025)
- [FMA: A Dataset For Music Analysis](https://mlsanity.com/paper/1612.01840/) (2016)
- [Stable Audio Open](https://mlsanity.com/paper/2407.14358/) (2024)
- [SoundStorm: Efficient Parallel Audio Generation](https://mlsanity.com/paper/2305.09636/) (2023)
- [Adapting Frechet Audio Distance for Generative Music Evaluation](https://mlsanity.com/paper/2311.01616/) (2023)
- [Long-form music generation with latent diffusion](https://mlsanity.com/paper/2404.10301/) (2024)
- [Dynamic Chunking for End-to-End Hierarchical Sequence Modeling](https://mlsanity.com/paper/2507.07955/) (2025)
- [The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation](https://mlsanity.com/paper/2311.10057/) (2023)
- [Moisesdb: A dataset for source separation beyond 4-stems](https://mlsanity.com/paper/2307.15913/) (2023)
- [EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer](https://mlsanity.com/paper/2409.10819/) (2024)
- [MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Music](https://mlsanity.com/paper/2402.09871/) (2024)
- [Continuous Audio Language Models](https://mlsanity.com/paper/2509.06926/) (2025)
- [FlexiCodec: A Dynamic Neural Audio Codec for Low Frame Rates](https://mlsanity.com/paper/2510.00981/) (2025)
- [Variable Bitrate Residual Vector Quantization for Audio Coding](https://mlsanity.com/paper/2410.06016/) (2024)
- [Unlocking Temporal Flexibility: Neural Speech Codec with Variable Frame Rate](https://mlsanity.com/paper/2505.16845/) (2025)
- [CodecSlime: Temporal Redundancy Compression of Neural Speech Codec via Dynamic Frame Rate](https://mlsanity.com/paper/2506.21074/) (2025)
- [Re-Bottleneck: Latent Re-Structuring for Neural Audio Autoencoders](https://mlsanity.com/paper/2507.07867/) (2025)
- [SpectroStream: A Versatile Neural Codec for General Audio](https://mlsanity.com/paper/2508.05207/) (2025)
- [Say More with Less: Variable-Frame-Rate Speech Tokenization via Adaptive Clustering and Implicit Duration Coding](https://mlsanity.com/paper/2509.04685/) (2025)

## Abstract (arXiv)

Neural audio autoencoders have become a core component of compression, feature extraction, and generation. However, while existing systems support variable bitrate, the vast majority of models still operate at a fixed latent frame-rate, allocating equal temporal budget to regions with very different information density, which can result in unnecessarily long sequences. We introduce Elastic Time, a dynamic frame-rate bottleneck that converts fixed-frame-rate autoencoders to dynamic ones. Our method learns a lightweight latent predictor used to decide which frames can be skipped and later reconstructed, enabling efficient greedy boundary selection at inference. Experiments show our method enables deployment-time rate control while improving efficiency-quality tradeoffs relative to baselines. Overall, we provide a flexible mechanism for adjusting temporal resolution in audio autoencoders, potentially facilitating more efficient downstream modeling for generation and long-context tasks.