# HILCodec: High-Fidelity and Lightweight Neural Audio Codec

arXiv: https://arxiv.org/abs/2405.04752 · HTML: https://mlsanity.com/paper/2405.04752/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2024.

## Built on by (most cited first, 9 of 9)

- [WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling](https://mlsanity.com/paper/2408.16532/) (2024)
- [WavChat: A Survey of Spoken Dialogue Models](https://mlsanity.com/paper/2411.13577/) (2024)
- [Recent Advances in Discrete Speech Tokens: A Review](https://mlsanity.com/paper/2502.06490/) (2025)
- [ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Control](https://mlsanity.com/paper/2406.01205/) (2024)
- [FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal Distillation](https://mlsanity.com/paper/2509.16195/) (2025)
- [SAME: A Semantically-Aligned Music Autoencoder](https://mlsanity.com/paper/2605.18613/) (2026)
- [U-Codec: Ultra Low Frame-rate Neural Speech Codec for Fast High-fidelity Speech Generation](https://mlsanity.com/paper/2510.16718/) (2025)
- [ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding](https://mlsanity.com/paper/2609.11642/) (2026)
- [TokenMapper: A Step Toward Interoperable Speech Token Translation](https://mlsanity.com/paper/2609.12563/) (2026)

## Abstract (arXiv)

The recent advancement of end-to-end neural audio codecs enables compressing audio at very low bitrates while reconstructing the output audio with high fidelity. Nonetheless, such improvements often come at the cost of increased model complexity. In this paper, we identify and address the problems of existing neural audio codecs. We show that the performance of the SEANet-based codec does not increase consistently as the network depth increases. We analyze the root cause of such a phenomenon and suggest a variance-constrained design. Also, we reveal various distortions in previous waveform domain discriminators and propose a novel distortion-free discriminator. The resulting model, HILCodec, is a real-time streaming audio codec that demonstrates state-of-the-art quality across various bitrates and audio types.