# EMO-Codec: An In-Depth Look at Emotion Preservation capacity of Legacy and Neural Codec Models With Subjective and Objective Evaluations

arXiv: https://arxiv.org/abs/2407.15458 · HTML: https://mlsanity.com/paper/2407.15458/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2024.

## Built on by (most cited first, 8 of 8)

- [Recent Advances in Discrete Speech Tokens: A Review](https://mlsanity.com/paper/2502.06490/) (2025)
- [Fake-Mamba: Real-Time Speech Deepfake Detection Using Bidirectional Mamba as Self-Attention's Alternative](https://mlsanity.com/paper/2508.09294/) (2025)
- [Segmentation-Variant Codebooks for Preservation of Paralinguistic and Prosodic Information](https://mlsanity.com/paper/2505.15667/) (2025)
- [Meta-PerSER: Few-Shot Listener Personalized Speech Emotion Recognition via Meta-learning](https://mlsanity.com/paper/2505.16220/) (2025)
- [EMO-TTA: Improving Test-Time Adaptation of Audio-Language Models for Speech Emotion Recognition](https://mlsanity.com/paper/2509.25495/) (2025)
- [Recovering Performance in Speech Emotion Recognition from Discrete Tokens via Multi-Layer Fusion and Paralinguistic Feature Integration](https://mlsanity.com/paper/2601.17085/) (2026)
- [End-to-End Training for Discrete Token LLM based TTS System](https://mlsanity.com/paper/2606.09234/) (2026)
- [Iterative Self-Learning for Expressive Text-to-Speech Synthesis](https://mlsanity.com/paper/2608.15910/) (2026)

## Abstract (arXiv)

The neural codec model reduces speech data transmission delay and serves as the foundational tokenizer for speech language models (speech LMs). Preserving emotional information in codecs is crucial for effective communication and context understanding. However, there is a lack of studies on emotion loss in existing codecs. This paper evaluates neural and legacy codecs using subjective and objective methods on emotion datasets like IEMOCAP. Our study identifies which codecs best preserve emotional information under various bitrate scenarios. We found that training codec models with both English and Chinese data had limited success in retaining emotional information in Chinese. Additionally, resynthesizing speech through these codecs degrades the performance of speech emotion recognition (SER), particularly for emotions like sadness, depression, fear, and disgust. Human listening tests confirmed these findings. This work guides future speech technology developments to ensure new codecs maintain the integrity of emotional information in speech.