mlsanityData to 7 Oct 2026

AudioDec: An Open-source Streaming High-fidelity Neural Audio Codec

arXiv 2305.16608 ↗First posted 26 May 2023eess.AScited by 47

Not on the map. Why?

Abstract (arXiv)

A good audio codec for live applications such as telecommunication is characterized by three key properties: (1) compression, i.e. the bitrate that is required to transmit the signal should be as low as possible; (2) latency, i.e. encoding and decoding the signal needs to be fast enough to enable communication without or with only minimal noticeable delay; and (3) reconstruction quality of the signal. In this work, we propose an open-source, streamable, and real-time neural audio codec that achieves strong performance along all three axes: it can reconstruct highly natural sounding 48~kHz speech signals while operating at only 12~kbps and running with less than 6~ms (GPU)/10~ms (CPU) latency. An efficient training paradigm is also demonstrated for developing such neural audio codecs for real-world scenarios. Both objective and subjective evaluations using the VCTK corpus are provided. To sum up, AudioDec is a well-developed plug-and-play benchmark for audio codec applications.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 May 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 17; 2025: 19; 2026: 7.

47 papers.

Sort:

By area:

  1. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  2. 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 124
  3. 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 119
  4. 2025MiMo-Audio: Audio Language Models are Few-Shot Learnerscited by 58
  5. 2024Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversationscited by 45
  6. 2023FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codeccited by 44
  7. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
  8. 2025Kling-Foley: Multimodal Diffusion Transformer for High-Quality Video-to-Audio Generationcited by 38
  9. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
  10. 2024DASB - Discrete Audio and Speech Benchmarkcited by 32
  11. 2024The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audiocited by 31
  12. 2023RepCodec: A Speech Representation Codec for Speech Tokenizationcited by 27
35 more
  1. 2023AV-RIR: Audio-Visual Room Impulse Response Estimationcited by 20
  2. 2024ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Controlcited by 17
  3. 2025FlowDec: A flow-based full-band general audio codec with high perceptual qualitycited by 15
  4. 2024SSDM: Scalable Speech Dysfluency Modelingcited by 13
  5. 2025Audio Super-Resolution with Latent Bridge Modelscited by 13
  6. 2026MOSS-TTS Technical Reportcited by 10
  7. 2024Codec-ASR: Training Performant Automatic Speech Recognition Systems with Discrete Speech Representationscited by 8
  8. 2024Apollo: Band-sequence Modeling for High-Quality Audio Restorationcited by 8
  9. 2024DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Modelscited by 8
  10. 2025FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal Distillationcited by 8
  11. 2023Towards General-Purpose Text-Instruction-Guided Voice Conversioncited by 6
  12. 2025PAST: Phonetic-Acoustic Speech Tokenizercited by 6
  13. 2024A Survey of Deep Learning Audio Generation Methodscited by 5
  14. 2024Codecfake: An Initial Dataset for Detecting LLM-based Deepfake Audiocited by 5
  15. 2025Conan: A Chunkwise Online Network for Zero-Shot Adaptive Voice Conversioncited by 4
  16. 2024PSCodec: A Series of High-Fidelity Low-bitrate Neural Speech Codecs Leveraging Prompt Encoderscited by 3
  17. 2024Continuous Speech Tokenizer in Text To Speechcited by 3
  18. 2024PolySpeech: Exploring Unified Multitask Speech Models for Competitiveness with Single-task Modelscited by 2
  19. 2025Measuring the Robustness of Audio Deepfake Detection under Real-World Corruptioncited by 2
  20. 2025Analysing the Language of Neural Audio Codecscited by 2
  21. 2025Task and Perception-aware Distributed Source Coding for Correlated Speech under Bandwidth-constrained Channelscited by 1
  22. 2025On the Design of Diffusion-based Neural Speech Codecscited by 1
  23. 2025Universal Semantic Disentangled Privacy-preserving Speech Representation Learningcited by 1
  24. 2025U-Codec: Ultra Low Frame-rate Neural Speech Codec for Fast High-fidelity Speech Generationcited by 1
  25. 2025Speech Recognition Model Improves Text-to-Speech Synthesis using Fine-Grained Rewardcited by 1
  26. 2026EuleroDec: A Complex-Valued RVQ-VAE for Efficient and Robust Audio Codingcited by 1
  27. 2026Language-Statistical Analysis of Neural Audio Codec Tokens Across Architectures, Corpora, and Noise Conditionscited by 1
  28. 2025Towards Reliable Audio Deepfake Attribution and Model Recognition: A Multi-Level Autoencoder-Based Frameworkcited by 0
  29. 2025Assessing speech quality metrics for evaluation of neural audio codecs under clean speech conditionscited by 0
  30. 2025Two-Dimensional Quantization for Geometry-Aware Audio Codingcited by 0
  31. 2025Decoder-Side Semantic Conditioning for Low-Bitrate Neural Speech Compressioncited by 0
  32. 2026SqueezeComposer: Temporal Speed-up is A Simple Trick for Long-form Music Composingcited by 0
  33. 2026Content is What Remains: Invariant Speech Tokenization from Parallel Utterancescited by 0
  34. 2026ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Codingcited by 0
  35. 2026TokenMapper: A Step Toward Interoperable Speech Token Translationcited by 0