mlsanityData to 14 Oct 2026

SNAC: Multi-Scale Neural Audio Codec

arXiv 2410.14411 ↗First posted 18 Oct 2024cs.SD cs.LG eess.AScited by 43

Not on the map. Why?

Abstract (arXiv)

Neural audio codecs have recently gained popularity because they can represent audio signals with high fidelity at very low bitrates, making it feasible to use language modeling approaches for audio generation and understanding. Residual Vector Quantization (RVQ) has become the standard technique for neural audio compression using a cascade of VQ codebooks. This paper proposes the Multi-Scale Neural Audio Codec, a simple extension of RVQ where the quantizers can operate at different temporal resolutions. By applying a hierarchy of quantizers at variable frame rates, the codec adapts to the audio structure across multiple timescales. This leads to more efficient compression, as demonstrated by extensive objective and subjective evaluations. The code and model weights are open-sourced at https://github.com/hubertsiuzdak/snac.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 18 Oct 2024.

Builds on

18 of 28 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Neural Discrete Representation Learningcited by 4,712
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  4. 2020Longformer: The Long-Document Transformercited by 3,413
  5. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  7. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  8. 2022High Fidelity Neural Audio Compressioncited by 687
  9. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  10. 2020Jukebox: A Generative Model for Musiccited by 542
  11. 2023Simple and Controllable Music Generationcited by 488
  12. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
6 more
  1. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 251
  2. 2018A Hierarchical Latent Vector Model for Learning Long-Term Structure in Musiccited by 131
  3. 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 74
  4. 2018The challenge of realistic music generation: modelling raw audio at scalecited by 63
  5. 2019Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decodercited by 49
  6. 2017End-to-End Optimized Speech Coding with Deep Neural Networkscited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 20; 2026: 20.

43 papers.

Sort:

By area:

  1. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 118
  2. 2025Step-Audio 2 Technical Reportcited by 105
  3. 2024WavChat: A Survey of Spoken Dialogue Modelscited by 90
  4. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 43
  5. 2025VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Modelcited by 31
  6. 2024From Specific-MLLMs to Omni-MLLMs: A Survey on MLLMs Aligned with Multi-modalitiescited by 22
  7. 2025Mini-Omni-Reasoner: Token-Level Thinking-in-Speaking in Large Speech Modelscited by 21
  8. 2024Lyra: An Efficient and Speech-Centric Framework for Omni-Cognitioncited by 20
  9. 2025FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networkscited by 18
  10. 2025TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modelingcited by 16
  11. 2025WavReward: Spoken Dialogue Models With Generalist Reward Evaluatorscited by 15
  12. 2025MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generationcited by 7
31 more
  1. 2025SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codeccited by 5
  2. 2025Unlocking Temporal Flexibility: Neural Speech Codec with Variable Frame Ratecited by 4
  3. 2025Robust and Efficient Autoregressive Speech Synthesis with Dynamic Chunk-wise Prediction Policycited by 4
  4. 2026SonoEdit: Null-Space Constrained Knowledge Editing for Pronunciation Correction in LLM-Based TTScited by 3
  5. 2026M*: A Modular, Extensible, Serving System for Multimodal Modelscited by 3
  6. 2025Spectrogram Patch Codec: A 2D Block-Quantized VQ-VAE and HiFi-GAN for Neural Speech Codingcited by 2
  7. 2026UniSRCodec: Unified and Low-Bitrate Single Codebook Codec with Sub-Band Reconstructioncited by 2
  8. 2025MARS6: A Small and Robust Hierarchical-Codec Text-to-Speech Modelcited by 1
  9. 2025StarVC: A Unified Auto-Regressive Framework for Joint Text and Speech Generation in Voice Conversioncited by 1
  10. 2025DeepOmni: Towards Seamless and Smart Speech Interaction with Adaptive Modality-Specific MoEcited by 1
  11. 2026Latent-Mark: An Audio Watermark Robust to Neural Codec Compressioncited by 1
  12. 2026VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singingcited by 1
  13. 2026NEUROTOKEN: Joint Source and Directional AAD with Envelope Decoding via Conditional Flow Matchingcited by 1
  14. 2025As Good as It KAN Get: High-Fidelity Audio Representationcited by 0
  15. 2025Assessing speech quality metrics for evaluation of neural audio codecs under clean speech conditionscited by 0
  16. 2025Learning Relationships Between Separate Audio Tracks for Creative Applicationscited by 0
  17. 2025Adapting Neural Audio Codecs to EEGcited by 0
  18. 2026Improving X-Codec-2.0 for Multi-Lingual Speech: 25 Hz Latent Rate and 24 kHz Samplingcited by 0
  19. 2026Audio Deepfake Detection in the Age of Advanced Text-to-Speech modelscited by 0
  20. 2026SqueezeComposer: Temporal Speed-up is A Simple Trick for Long-form Music Composingcited by 0
  21. 2026LoSATok: Low-Dimensional Semantic-Acoustic Tokenizer for Cross-Domain Audio Understanding and Generationcited by 0
  22. 2026Probing Low Frame Rate Degradation in Neural Audio Codecscited by 0
  23. 2026NAC: Neural Action Codec for Vision-Language-Action Modelscited by 0
  24. 2026MusicMark: A Robust Generative Watermarking Framework for Music Generationcited by 0
  25. 2026A Training-Free Proactive Defense Against Partial Speech Manipulation via Self-Embedding Steganographycited by 0
  26. 2026Combining Self-Embedding Audio Watermarking with Ultra-Low-Bitrate Neural Codecscited by 0
  27. 2026TokenMapper: A Step Toward Interoperable Speech Token Translationcited by 0
  28. 2026LACE: Layer-Wise Compression for Dynamic Frame Rate Codecscited by 0
  29. 2026Tokens Change, Structure Endures: Spectral Watermarking for Generated Speechcited by 0
  30. 2026Controlling Speaking Rate in Autoregressive TTS via Activation Steeringcited by 0
  31. 2026GS-Codec: A Gaussian-Splatting Bottleneck for Neural Audio Codingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Variable Bitrate Residual Vector Quantization for Audio Coding
  2. 2025QINCODEC: Neural Audio Compression with Implicit Neural Codebooks
  3. 2025Finite Scalar Quantization Enables Redundant and Transmission-Robust Neural Audio Compression at Low Bit-rates
  4. 2026Geometric Iterative Retrieval for Neural Audio Codec Resynthesis
  5. 2026Switchcodec: Adaptive residual-expert sparse quantization for high-fidelity neural audio coding
  6. 2026Entropy-Guided GRVQ for Ultra-Low Bitrate Neural Speech Codec ↗
  7. 2022Cross-Scale Vector Quantization for Scalable Neural Speech Coding
  8. 2024APCodec: A Neural Audio Codec with Parallel Amplitude and Phase Spectrum Encoding and Decoding