# EASY: Emotion-aware Speaker Anonymization via Factorized Distillation

arXiv: https://arxiv.org/abs/2505.15004 · HTML: https://mlsanity.com/paper/2505.15004/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2025.

## Built on by (most cited first, 6 of 6)

- [The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization](https://mlsanity.com/paper/2601.11846/) (2026)
- [Stream-Voice-Anon: Enhancing Utility of Real-Time Speaker Anonymization via Neural Audio Codec and Language Models](https://mlsanity.com/paper/2601.13948/) (2026)
- [SegReConcat: A Data Augmentation Method for Voice Anonymization Attack](https://mlsanity.com/paper/2508.18907/) (2025)
- [StreamVoiceAnon+: Emotion-Preserving Streaming Speaker Anonymization via Frame-Level Acoustic Distillation](https://mlsanity.com/paper/2603.06079/) (2026)
- [DAST: A Dual-Stream Voice Anonymization Attacker with Staged Training](https://mlsanity.com/paper/2603.12840/) (2026)
- [LiveAnon: Improving Privacy and Utility in Streaming Speaker Anonymization with Codec Language Models](https://mlsanity.com/paper/2610.15379/) (2026)

## Abstract (arXiv)

Emotion plays a significant role in speech interaction, conveyed through tone, pitch, and rhythm, enabling the expression of feelings and intentions beyond words to create a more personalized experience. However, most existing speaker anonymization systems employ parallel disentanglement methods, which only separate speech into linguistic content and speaker identity, often neglecting the preservation of the original emotional state. In this study, we introduce EASY, an emotion-aware speaker anonymization framework. EASY employs a novel sequential disentanglement process to disentangle speaker identity, linguistic content, and emotional representation, modeling each speech attribute in distinct subspaces through a factorized distillation approach. By independently constraining speaker identity and emotional representation, EASY minimizes information leakage, enhancing privacy protection while preserving original linguistic content and emotional state. Experimental results on the VoicePrivacy Challenge official datasets demonstrate that our proposed approach outperforms all baseline systems, effectively protecting speaker privacy while maintaining linguistic content and emotional state.