mlsanityData to 14 Oct 2026

NanoCodec: Towards High-Quality Ultra Fast Speech LLM Inference

arXiv 2508.05835 ↗First posted 7 Aug 2025eess.AS cs.CL cs.SDcited by 15

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have significantly advanced audio processing by leveraging audio codecs to discretize audio into tokens, enabling the application of language modeling techniques to speech data. However, existing audio codecs often operate at high frame rates, leading to slow training and inference, particularly for autoregressive models. To address this, there is growing interest in low frame-rate audio codecs, which reduce the number of autoregressive steps required to generate one second of audio. In this paper, we conduct ablation studies to examine the impact of frame rate, bitrate, and causality on codec reconstruction quality. Based on our findings, we introduce NanoCodec, a state-of-the-art audio codec that achieves high-quality compression at just 12.5 frames per second (FPS). NanoCodec outperforms related works across various bitrate ranges, establishing a new benchmark for low-latency and efficient Speech LLM training and inference.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 29 of its 36 references that are in the map. First posted on arXiv 7 Aug 2025.

Builds on

29 of 36 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  3. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  4. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  5. 2022High Fidelity Neural Audio Compressioncited by 687
  6. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  7. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  8. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  9. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 440
  10. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  11. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  12. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
17 more
  1. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 251
  2. 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 129
  3. 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 100
  4. 2021TitaNet: Neural Model for speaker representation with 1D Depth-wise separable convolutions and global contextcited by 62
  5. 2021Hi-Fi Multi-Speaker English TTS Datasetcited by 60
  6. 2023TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudiocited by 59
  7. 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 58
  8. 2024Scaling Transformers for Low-Bitrate High-Quality Speech Codingcited by 49
  9. 2020Clova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020cited by 36
  10. 2025Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidancecited by 27
  11. 2024TS3-Codec: Transformer-Based Simple Streaming Single Codeccited by 25
  12. 2024ECAPA2: A Hybrid Neural Network Architecture and Training Strategy for Robust Speaker Embeddingscited by 22
  13. 2022Textless Direct Speech-to-Speech Translation with Discrete Speech Representationcited by 19
  14. 2024APCodec: A Neural Audio Codec with Parallel Amplitude and Phase Spectrum Encoding and Decodingcited by 13
  15. 2024Low Frame-rate Speech Codec: a Codec Designed for Fast High-quality Speech LLM Training and Inferencecited by 11
  16. 2024Gull: A Generative Multifunctional Audio Codeccited by 5
  17. 2024An Intra-BRNN and GB-RVQ Based END-TO-END Neural Audio Codeccited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 11.

15 papers.

Sort:

By area:

  1. 2025Chronological Thinking in Full-Duplex Spoken Dialogue Language Modelscited by 14
  2. 2025FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal Distillationcited by 9
  3. 2026F-Actor: Controllable Conversational Behaviour in Full-Duplex Modelscited by 3
  4. 2026Reconstruct! Don't Encode: Self-Supervised Representation Reconstruction Loss for High-Intelligibility and Low-Latency Streaming Neural Audio Codeccited by 2
  5. 2025Align2Speak: Improving TTS for Low Resource Languages via ASR-Guided Online Preference Optimizationcited by 1
  6. 2026MagpieTTS-LF: Inference-Time Long-Form Speech Generation Without Training on Long-Form datacited by 1
  7. 2025Frame-Stacked Local Transformers For Efficient Multi-Codebook Speech Generationcited by 0
  8. 2026IRAF: Interference-Resilient Adaptive Fusion for Noise-Robust End-to-End Full-Duplex Spoken Dialogue Systemscited by 0
  9. 2026Probing Low Frame Rate Degradation in Neural Audio Codecscited by 0
  10. 2026OLIVE: View-Augmented Latent Prediction with Waveform Reconstruction for Speech SSLcited by 0
  11. 2026ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Codingcited by 0
  12. 2026TokenMapper: A Step Toward Interoperable Speech Token Translationcited by 0
3 more
  1. 2026Controlling Backchannels in Streamable Full-duplex Modelscited by 0
  2. 2026All In Good Time: Causality-Aware Framework for LLM-Based Simultaneous Speech-to-Speech Translationcited by 0
  3. 2026RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Cancited by 0