# Codec-SUPERB @ SLT 2024: A lightweight benchmark for neural audio codec models

arXiv: https://arxiv.org/abs/2409.14085 · HTML: https://mlsanity.com/paper/2409.14085/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2024.

## Built on by (most cited first, 7 of 7)

- [On The Landscape of Spoken Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.08528/) (2025)
- [Discrete Speech Unit Extraction via Independent Component Analysis](https://mlsanity.com/paper/2501.06562/) (2025)
- [LM-SPT: LM-Aligned Semantic Distillation for Speech Tokenization](https://mlsanity.com/paper/2506.16738/) (2025)
- [Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Tasks](https://mlsanity.com/paper/2601.12205/) (2026)
- [Towards Generalized Source Tracing for Codec-Based Deepfake Speech](https://mlsanity.com/paper/2506.07294/) (2025)
- [Representing Speech Through Autoregressive Prediction of Cochlear Tokens](https://mlsanity.com/paper/2508.11598/) (2025)
- [How Robust Are Neural Audio Codecs for African Speech? A Multi-Task Benchmark and the Limits of Perceptual Quality](https://mlsanity.com/paper/2610.00154/) (2026)

## Abstract (arXiv)

Neural audio codec models are becoming increasingly important as they serve as tokenizers for audio, enabling efficient transmission or facilitating speech language modeling. The ideal neural audio codec should maintain content, paralinguistics, speaker characteristics, and audio information even at low bitrates. Recently, numerous advanced neural codec models have been proposed. However, codec models are often tested under varying experimental conditions. As a result, we introduce the Codec-SUPERB challenge at SLT 2024, designed to facilitate fair and lightweight comparisons among existing codec models and inspire advancements in the field. This challenge brings together representative speech applications and objective metrics, and carefully selects license-free datasets, sampling them into small sets to reduce evaluation computation costs. This paper presents the challenge's rules, datasets, five participant systems, results, and findings.