# StreamVoiceAnon+: Emotion-Preserving Streaming Speaker Anonymization via Frame-Level Acoustic Distillation

arXiv: https://arxiv.org/abs/2603.06079 · HTML: https://mlsanity.com/paper/2603.06079/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 34 references that are in the map.
- First posted on arXiv 6 Mar 2026.

## Builds on (most cited first, 27 of 27)

- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation](https://mlsanity.com/paper/2312.15185/) (2023)
- [Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings](https://mlsanity.com/paper/2104.03502/) (2021)
- [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://mlsanity.com/paper/2110.01900/) (2021)
- [The VoicePrivacy 2020 Challenge: Results and findings](https://mlsanity.com/paper/2109.00648/) (2021)
- [Speaker anonymisation using the McAdams coefficient](https://mlsanity.com/paper/2011.01130/) (2020)
- [Design Choices for X-vector Based Speaker Anonymization](https://mlsanity.com/paper/2005.08601/) (2020)
- [Speaker anonymization using neural audio codec language models](https://mlsanity.com/paper/2309.14129/) (2023)
- [Anonymizing Speech with Generative Adversarial Networks to Preserve Speaker Privacy](https://mlsanity.com/paper/2210.07002/) (2022)
- [The VoicePrivacy 2022 Challenge: Progress and Perspectives in Voice Anonymisation](https://mlsanity.com/paper/2407.11516/) (2024)
- [NPU-NTU System for Voice Privacy 2024 Challenge](https://mlsanity.com/paper/2409.04173/) (2024)
- [Gradient Remedy for Multi-Task Learning in End-to-End Noise-Robust Speech Recognition](https://mlsanity.com/paper/2302.11362/) (2023)
- [Privacy versus Emotion Preservation Trade-offs in Emotion-Preserving Speaker Anonymization](https://mlsanity.com/paper/2409.03655/) (2024)
- [StreamVoice: Streamable Context-Aware Language Modeling for Real-time Zero-Shot Voice Conversion](https://mlsanity.com/paper/2401.11053/) (2024)
- [End-to-end streaming model for low-latency speech anonymization](https://mlsanity.com/paper/2406.09277/) (2024)
- [GenVC: Self-Supervised Zero-Shot Voice Conversion](https://mlsanity.com/paper/2502.04519/) (2025)
- [The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization](https://mlsanity.com/paper/2601.11846/) (2026)
- [DarkStream: real-time speech anonymization with low latency](https://mlsanity.com/paper/2509.04667/) (2025)
- [Stream-Voice-Anon: Enhancing Utility of Real-Time Speaker Anonymization via Neural Audio Codec and Language Models](https://mlsanity.com/paper/2601.13948/) (2026)
- [NTU-NPU System for Voice Privacy 2024 Challenge](https://mlsanity.com/paper/2410.02371/) (2024)
- [EASY: Emotion-aware Speaker Anonymization via Factorized Distillation](https://mlsanity.com/paper/2505.15004/) (2025)
- [TVTSyn: Content-Synchronous Time-Varying Timbre for Streaming Voice Conversion and Anonymization](https://mlsanity.com/paper/2602.09389/) (2026)
- [StarGAN-VC++: Towards Emotion Preserving Voice Conversion Using Deep Embeddings](https://mlsanity.com/paper/2309.07592/) (2023)
- [Adapting General Disentanglement-Based Speaker Anonymization for Enhanced Emotion Preservation](https://mlsanity.com/paper/2408.05928/) (2024)
- [Use Cases for Voice Anonymization](https://mlsanity.com/paper/2508.06356/) (2025)
- [Emo-StarGAN: A Semi-Supervised Any-to-Many Non-Parallel Emotion-Preserving Voice Conversion](https://mlsanity.com/paper/2309.07586/) (2023)
- [Why disentanglement-based speaker anonymization systems fail at preserving emotions?](https://mlsanity.com/paper/2501.13000/) (2025)

## Built on by (most cited first, 1 of 1)

- [LiveAnon: Improving Privacy and Utility in Streaming Speaker Anonymization with Codec Language Models](https://mlsanity.com/paper/2610.15379/) (2026)

## Abstract (arXiv)

We address the challenge of preserving emotional content in streaming speaker anonymization (SA). Neural audio codec language models trained for audio continuation tend to degrade source emotion: content tokens discard emotional information, and the model defaults to dominant acoustic patterns rather than preserving paralinguistic attributes. We propose supervised finetuning with neutral-emotion utterance pairs from the same speaker, combined with frame-level emotion distillation on acoustic token hidden states. All modifications are confined to finetuning, which takes less than 2 hours on 4 GPUs and adds zero inference latency overhead, while maintaining a competitive 180ms streaming latency. On the VoicePrivacy 2024 protocol, our approach achieves a 49.2% UAR (emotion preservation) with 5.77% WER (intelligibility), a +24% relative UAR improvement over the baseline (39.7%->49.2%) and +10% over the emotion-prompt variant (44.6% UAR), while maintaining strong privacy (EER 49.0%). Demo and code are available: https://anonymous3842031239.github.io/