# User-Level Differential Privacy against Attribute Inference Attack of Speech Emotion Recognition in Federated Learning

arXiv: https://arxiv.org/abs/2204.02500 · HTML: https://mlsanity.com/paper/2204.02500/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2022.

## Built on by (most cited first, 9 of 9)

- [FedMultimodal: A Benchmark For Multimodal Federated Learning](https://mlsanity.com/paper/2306.09486/) (2023)
- [A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness](https://mlsanity.com/paper/2212.09006/) (2022)
- [Gibberish is All You Need for Membership Inference Detection in Contrastive Language-Audio Pretraining](https://mlsanity.com/paper/2410.18371/) (2024)
- [Differential privacy enables fair and accurate AI-based analysis of speech disorders while protecting patient data](https://mlsanity.com/paper/2409.19078/) (2024)
- [From Centralized to Decentralized Federated Learning: Theoretical Insights, Privacy Preservation, and Robustness Challenges](https://mlsanity.com/paper/2503.07505/) (2025)
- [EMO-Debias: Benchmarking Gender Debiasing Techniques in Multi-Label Speech Emotion Recognition](https://mlsanity.com/paper/2506.04652/) (2025)
- [Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence](https://mlsanity.com/paper/2407.14710/) (2024)
- [VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks](https://mlsanity.com/paper/2509.18413/) (2025)
- [Exploring Audio Editing Features as User-Centric Privacy Defenses Against Large Language Model(LLM) Based Emotion Inference Attacks](https://mlsanity.com/paper/2501.18727/) (2025)

## Abstract (arXiv)

Many existing privacy-enhanced speech emotion recognition (SER) frameworks focus on perturbing the original speech data through adversarial training within a centralized machine learning setup. However, this privacy protection scheme can fail since the adversary can still access the perturbed data. In recent years, distributed learning algorithms, especially federated learning (FL), have gained popularity to protect privacy in machine learning applications. While FL provides good intuition to safeguard privacy by keeping the data on local devices, prior work has shown that privacy attacks, such as attribute inference attacks, are achievable for SER systems trained using FL. In this work, we propose to evaluate the user-level differential privacy (UDP) in mitigating the privacy leaks of the SER system in FL. UDP provides theoretical privacy guarantees with privacy parameters ε and δ. Our results show that the UDP can effectively decrease attribute information leakage while keeping the utility of the SER system with the adversary accessing one model update. However, the efficacy of the UDP suffers when the FL system leaks more model updates to the adversary. We make the code publicly available to reproduce the results in https://github.com/usc-sail/fed-ser-leakage.